📢 #acl2025 - main: 🤔Continued pretraining of LLMs in new languages often includes English data, but why? 💡We found English inclusion doesn't improve valid perplexity in the target language, yet critical for the emergence of abilities such as in-context learning! (1/5)

Can you train a performant language models without using unlicensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1&2

We are launching a new speaker series at EleutherAI, focused on promoting recent research by our team and community members. Our first talk is by Catherine Arnett on tokenizers, their limitations, and how to improve them.

Excited to announce the call for papers for the Multilingual Representation Learning workshop #EMNLP2025 sigtyp.github.io/ws2025-mrl.html with Duygu Ataman Catherine Arnett Jiayi Wang Fabian David Schmidt Tyler Chang Hila Gonen and amazing speakers: Alice Oh, Kelly Marchisio, & Pontus Stenetorp

Can coding agents autonomously implement AI research extensions? We introduce RExBench, a benchmark that tests if a coding agent can implement a novel experiment based on existing research and code. Finding: Most agents we tested had a low success rate, but there is promise!

Check out our new paper: “How Do Vision-Language Models Process Conflicting Information Across Modalities?”! Vision-language models often struggle with conflicting inputs - we show how their internal representations and key attention heads reveal when and how this happens, and

Can data owners & LM developers collaborate to build a strong shared model while each retaining data control? Introducing FlexOlmo💪, a mixture-of-experts LM enabling: • Flexible training on your local data without sharing it • Flexible inference to opt in/out your data

Sad to miss ACL in Vienna but so many of our members of SEACrowd are going to be there to present this work🔥 Reach out or find us in our merch 😉 Learn about our ongoing cool initiatives and how to participate or get our merch 😎

Looking forward to attending #cogsci2025! I’m especially excited to meet students who will be applying to PhD programs in Computational Ling/CogSci in the coming cycle. Please reach out if you want to meet up and chat! Email is best, but DM also works if you must quick🧵:

With a $20 million grant from the U.S. National Science Foundation, Brown University researchers will lead an artificial intelligence research institute aimed at developing a new generation of AI assistants for use in mental and behavioral health. brown.edu/news/2025-07-2…

Introducing Command A Vision, a state-of-the-art generative model that excels across multimodal image capabilities that matter for enterprises!