Getting labeled data for clinical NLP can be prohibitively difficult. In work at #emnlp2022, we find that GPT-3 models perform very well at clinical tasks in the few-shot setting, indicating a new paradigm for transforming EHR notes into actionable data news.mit.edu/2022/large-lan…

Can AI tell us when it can predict better than humans and when humans are better? In our AISTATS23 (oral) with MIT-IBM Watson AI Lab paper we build AI models that 1) complement humans and 2) can defer to humans when necessary arxiv.org/abs/2301.06197

🤩🤩🤩

🤩🤩🤩

Excited to announce what we've been up to this year!

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

Thanks AK for tweeting our work! 🤝Co-LLM trains a language model to collaboratively decode with other LMs. It decides whether to invoke the other assistant LM for each token. By optimally combining the expertise from each LM, co-llm generation can be better than using

So happy to defend my PhD thesis and couldn't have done it without a 1 of 1 advisor David Sontag and an incredible committee Eric Horvitz Arvind Satyanarayan Elena Glassman

The "primary area" field on NeurIPS Conference OpenReview has no good option for work on unsupervised or semi-supervised learning?

Can LLMs learn to "phone a friend?" 🧵 MIT CSAIL’s new "Co-LLM" algorithm can pair a general-purpose base LLM w/a more specialized model & help them work together. It reviews each token & sees where it needs to call upon an expert, leading to more accurate & efficient replies to

Why do we treat train and test times so differently? Why is one “training” and the other “in-context learning”? Just take a few gradients during test-time — a simple way to increase test time compute — and get a SoTA in ARC public validation set 61%=avg. human score! ARC Prize

I just wrote my first blog post in four years! It is called "Deriving Muon". It covers the theory that led to Muon and how, for me, Muon is a meaningful example of theory leading practice in deep learning (1/11)

anyway, none of this is possible because the data processing inequality

Thrilled to share that our Byte Latent Transformer won an Outstanding Paper Award at ACL 2025! 🏆