🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Can insights from synthetic experiments and interpretability lead to real improvements in language modeling? We: > propose a formal model for in-context learning > uncover "n-gram heads" = high order induction heads, crucial for ICLL > improve Transformer LM perplexity by 6.7%

I'm hiring a postdoc at UVA to edit and detoxify LLMs! This is an exciting opportunity to join a vibrant research community and to collaborate with Steven L. Johnson (he/him) and Maarten Sap (he/him) Feel free to get in touch and please help spread the word! tinyurl.com/yeywuu3b

Nowhere is safe in Gaza. Over 70% of civilian infrastructure has been destroyed or severely damaged. 84% of health facilities have been affected. Civilians & critical infrastructure are #NotATarget & must always be protected. ochaopt.org/content/hostil…

"I think we should kill them all." -Congressman Andy Ogles (R-TN) on Palestinians in Gaza x.com/i/status/17600…

I’m super excited to release our 100+ page collaborative agenda - led by Usman Anwar - on “Foundational Challenges In Assuring Alignment and Safety of LLMs” alongside 35+ co-authors from NLP, ML, and AI Safety communities! Some highlights below...

Constitutional AI showed LMs can learn to follow constitutions by labeling their own outputs. But why can't we just tell a base model the principles of desired behavior and rely on it to act appropriately? Introducing SAMI: Self-Supervised Alignment with Mutual Information!

Good teachers *adapt* to student beliefs & misconceptions: Can LLM teachers? In new work w/ Jacob Andreas, we introduce 1) the AdapT 👩‍🏫 (Adaptive Teaching) evaluation framework & 2) AToM ⚛️ (Adaptive Teaching tOwards Misconceptions), a new probabilistic teaching method. (1/n)

As the world changes, documents go out of date. How can we adapt RAG systems to a stream of changing world data? We introduce ERASE, a way of updating and propagating facts within knowledge bases, and CLARK, a dataset targeting these update problems arxiv.org/abs/2406.11830… 1/

I'll be in Vienna for ICML! I present what's missing in recent SSM/linear attention variants and a way to interpolate them towards Transformer. I'll be - at poster session on Wednesday, 11:30am - giving a talk on ICL workshop on Sunday, 9am

Hi ICML! We're presenting papers this week: - understanding how LMs learn new langs in-context arxiv.org/pdf/2401.12973 - agents for automated interpretability research: …imodal-interpretability.csail.mit.edu/maia/ - calibrating LMs by marginalizing over missing context: arxiv.org/abs/2311.08718

📝 Many projects use translated benchmarks, but issues like translation errors and cultural irrelevance persist. To solve this, we've created a native Turkish benchmark, TurkishMMLU, and a leaderboard for LLMs in various setups, including no-CoT, CoT, and by difficulty.

I am sad to miss #ACL2024 but DerryTWijaya will be presenting our work at the poster session today at 12.15 pm!

Why do we treat train and test times so differently? Why is one “training” and the other “in-context learning”? Just take a few gradients during test-time — a simple way to increase test time compute — and get a SoTA in ARC public validation set 61%=avg. human score! ARC Prize

With OpenAI o1, we developed one way to scale test-time compute, but it isn't the only way and might not be the best way. I'm excited to see academic researchers explore new approaches in this direction.

I am looking for Machine Learning Intern for the Spring or Summer terms at the AI Institute for scaling our robot policy learning stack. Apply here and DM me! jobs.lever.co/bostondynamics…

🚨 URGENT 🚨 For over 14 hours, the crew of the Freedom Flotilla has been illegally detained by Israel, following the equally unlawful interception of their humanitarian vessel as it was delivering aid to Gaza—outside Israeli territorial waters. ACTION 1: We need your help to