There are 12 days left to apply to the Predoctoral Young Investigator program with AllenNLP! Applications must be submitted by February 15th: boards.greenhouse.io/thealleninstit…

Pretrained LMs suffer under domain shift. How to adapt to new domains without extra training? Using an AdapterSoup! We propose averaging the weights of related domain adapters at test time. Lower perplexity across 10 eval domains. EACL camera-ready📜: arxiv.org/pdf/2302.07027…

🦙🐪🐫 So many instruction tuning datasets came out recently! How valuable are they, and how far are open models really from proprietary ones like ChatGPT? 🧐We did a systematic exploration, and built Tülu---a suite of LLaMa-tuned models up to 65B! 📜arxiv.org/abs/2306.04751

The models and code are out now! Code 💻: github.com/allenai/open-i… Tulu-65B 🐪: huggingface.co/allenai/tulu-6… All the others🐫: huggingface.co/models?other=a…

Check out our new 70B DPO model here: huggingface.co/allenai/tulu-2… AFAIK currently the best model on AlpacaEval with a public finetuning set! More details once the AI sphere calms down a bit... 😅

🥳We trained a 70B DPO Tülu-2 😮 For details on data, adaptation and QLoRA training check out the paper: arxiv.org/abs/2311.10702

Check out the Tulu 2 suite 🐪, a set of Llama-2 models finetuned+DPO-trained on a mixture of publicly available datasets! Our best-performing models are competitive with SoTA open models on a range of benchmarks incl. AlpacaEval and MT-Bench. 📜Paper: arxiv.org/abs/2311.10702

Walmart tech vet, AI2 research scientists are behind new Seattle startup Spiffy geekwire.com/2023/walmart-t…

🚀Exciting new results on #AlpacaEval Leaderboard! The PairRM (0.4B) greatly improves LLMs by inference-time alignment (i.e., best-of-N sampling; re-ranking)! Now we have a new open-source solution to match #GPT4's performance (on this particular benchmark). 💡The significant

Tulu-2-DPO-70B tops the charts for open models! Now to see how much better we can make Tulu 3 🫡

Conference week coming up, AGU (American Geophysical Union) Fall Meeting and NeurIPS Conference Climate Change AI workshop! AI2 Climate Modeling team members will be presenting at both. 1/

Updated version of AI2 Climate Emulator (ACE) paper with: -> 100 year run ⌚️ -> testing on unseen realistic SST dataset 🌍 -> code+data+checkpoint released github.com/ai2cm/ace arxiv.org/abs/2310.02074

Accepted to EACL! 😊

Hello OLMo! Congrats to the amazing Ai2 team! 7B params, 2T tokens, open training code, open data, intermediate checkpoints, Apache 2.0, the works. A giant leap for open science. Nicely done Mechanical Dirk, Iz Beltagy, Luca Soldaini 🎀, and so many others! blog.allenai.org/hello-olmo-a-t…

OLMo is here! And it’s 100% open. It’s a state-of-the-art LLM and we are releasing it with all pre-training data and code. Let’s get to work on understanding the science behind LLMs. Learn more about the framework and how to access it here: blog.allenai.org/olmo-open-lang…

OLMo-7b is finally out 🎉, and we are releasing everything; weights, intermediate checkpoints, training code and logs, training data and toolkit, evaluation and adaptation code and data. Most of it has been released, and the rest is coming soon. OLMo-65b and Adapted OLMo-7b are

We are excited to contribute to the launch of Allen Institute for AI’s OLMo model, a truly open-source, state-of-the-art language model. We’re looking forward to continuing to work together to drive forward open AI innovation.

Today we released OLMo and its 100% open! I'm incredibly proud of every single team member who worked tirelessly in the past few months to make this happen🏆 This release includes all the data, code, checkpoints, and more! #olmo Ai2

I can’t believe we wrote a 83 pages data paper, so sorry y’all

Allen AI presents OLMo Accelerating the Science of Language Models paper page: huggingface.co/papers/2402.00… a state-of-the-art, truly Open Language Model and its framework to build and study the science of language modeling. Unlike most prior efforts that have only released model