What are you excited to study with these new SAEs?

How do popular LM interventions like editing, compression, and unlearning interact? We study to what degree popular interventions are composable — a crucial requirement for their practical application relevant to factuality, safety, and efficiency. 🧵 arxiv.org/pdf/2407.06483

Tomorrow 11 am EDT / 8 am PDT I'll be discussing openness across the AI stack + building open datasets and the tech and policy challenges that come with it, based on two recent Mozilla workshops. Sign up here: eventbrite.co.uk/e/openuk-resea…

Mozilla The dataset work stems from the Dataset Convening, co-hosted by EleutherAI. Here's a blog post summarizing the event: blog.mozilla.org/en/mozilla/dat…

Circuit analysis is a common tool in mechanistic interpretability for understanding model behaviors when executing certain tasks. But how well do these findings generalize throughout model training or to models of different sizes?

Looking for EleutherAI at ICML Conference? Come meet our community and check out their fabulous work. Featuring (in order of appearance): Lintang Sutawika Hailey Schoelkopf Aviya Skowron Stella Biderman Guillaume "Vermeille" Sanchez Honglu Fan Dashiell Stander Qinan Yu Nora Belrose Curt Tigges

Catch our tutorial at ICML today at Lehar 1-4 from 3:30-5:30 pm!

As models become larger and more unwieldy, auto-interp methods have becoming increasingly important. We are excited to be releasing the most comprehensive auto interp library to enable wider research on this topic. github.com/EleutherAI/sae…

We were very happy with the reception to our researchers Lintang Sutawika and Hailey Schoelkopf 's ICML tutorial, "Challenges in LM Evaluation", this past week! For all those who requested it, the slides are now available at lm-evaluation-challenges.github.io . Enjoy!

Very excited for our community leader Kyle O'Brien who just had his first first-author paper accepted to TMLR!

Excellent work from the people who brought you Zambda, and co-authored by Quentin Anthony and our long-time community member nshepperd!

Glad to have it! We look forward to using this benchmark in our future research.

The RWKV v6 Finch lines of models are here Scaling from 1.6B all the way to 14B Pushing the boundary for an Attention-free transformer, and Multi-lingual models. Cleanly licensedm Apache 2, under The Linux Foundation Find out more from the writeup here: blog.rwkv.com/p/rwkv-v6-finc…