LLMs excel in math. Introducing a new benchmark, we observe: They struggle with creative and many-step questions (even with CoT), their performance varies widely even on similar topics, and they engage in genuine reasoning only in about half of cases. 1/n arxiv.org/abs/2406.05194

I'm at NAACL and will be presenting a poster this afternoon (4-5:30 pm) on our work, The Bias Amplification Paradox in Text-to-Image Generation. Do stop by and check out our work! #naacl2024 aclanthology.org/2024.naacl-lon…

Want to improve prompting? Come check out this exciting work at NAACL by Yasaman Razeghi@NAACL and Rajasekhar Mekala today at 2 pm!

Check out Tamanna’s awesome work at NAACL this week!

The new Hugging Face Ethics and Society Newsletter is here! I wrote with Avijit Ghosh about "Building Better AI: The Importance of Data Quality" on the impact of data quality, what to consider for responsible data practices and its impact. huggingface.co/blog/ethics-so…

Multilingual Safety is severely under researched for the many of languages that LLMs support. aakanksha has done incredible work on mitigating this--crucially on investigating whether training on global harms can help mitigate even culturally specific local harms. Read it 🔥

Checkout this awesome work led by Chantal about detecting syntactic templates! We Show how pretraining data distribution governs model behavior, how some patterns are more often used than others, and some cool bits about soft memorization detection!

I’ll be presenting this work at #ICML2024 next week! Stop by poster #2310 in Hall C 4-9 between 1:30-3 pm CEST on Thu, Jul 25 💜

We’re excited to announce our Series D financing to accelerate growth, expand our team, & develop our next class of frontier, enterprise-grade, data privacy-focused AI technology. We’re bringing highly scalable and secure AI solutions to global enterprises wherever their data is

Bruna Oewel and I are running a survey to understand how men perceive mental wellbeing and the potential of using technology for support. Take our survey here and share it with other people you know! uci.questionpro.com/tech-and-wellb…

Applications will be open until end of this month. Looking forward to hearing from all of you! 🔥🌎🌏🌍✨

Thrilled to have won the Best Social Impact Paper Award at #ACL2024 ACL 2024 for our work; DialectBench! Big thanks to all my amazing collaborators who made this possible!

A break from the usual #ACL2024 program: Preethi Seshadri 🔥is working with me on open research for fairness of LLMs in hiring. 👇 is a short form to share a resume so we can check our synth data is predictive of real data. Please help! 🙏💖 (no training/ sharing, we delete it after)

Reminder to share your resumes if you'd like to help our investigation! The form is super short and easy ✨

I am not focused on AGI or 🍓s nor do i think that focus would be helpful right now. building value > chasing sci-fi. We are not near having AGI, super intelligence, or digital gods anytime soon but models are already providing value and can add so much more.

Cohere is heading to Hack the North 2024, Canada's largest hackathon! We're hosting an API challenge with a workshop, offering a chance to win prizes. Meet our team, try our demo, and grab some swag.

📢📢 Updates to WIMBD search wimbd.apps.allenai.org 📢📢 - WIMBD is now on pypi, easily installed with pip - Search using `slop` which allows more flexibility with the search queries - Additional indices: -- dolma v1.7 -- tulu2 Ai2