Check out our new paper "Enabling Large Language Models to Generate Text with Citations", w/ Howard Yen, Jiatong Yu, and Danqi Chen. We study a new paradigm where LLMs generate text with citations. We propose ALCE, the first benchmark for Automatic LLMs’ Citation Evaluation.

Adapting Language Models to Compress Contexts by Chevalier et al. is a cool paper. 🧪 Compress long contexts into summary vectors, which are then used as “soft prompt” inputs into an LLM. You can concat a bunch of summary vectors to represent an entire document/set of documents

Learning Transformer Programs We designed a modified Transformer that can be trained to solve a task and then automatically converted into a discrete, human-readable program. With Alex Wettig and Danqi Chen. Paper: arxiv.org/abs/2306.01128 Code: github.com/princeton-nlp/… [1/12]

Strategies to ensure that AI tools are developed and used responsibly must be a fundamental part of the development process rather than an afterthought. How might this be achieved? Join our expert panel to discuss on June 6th. FREE registration here: ias.edu/events/steerin…

If you haven't signed up for the National Academies workshop on AI for Math, now is your chance! We have wonderful speakers like Moshe Vardi, Sean Welleck, Thierry Coquand, and Stella Biderman, and wonderful moderators like Yann LeCun, Terry Tao, and yours truly 😄 nationalacademies.org/our-work/ai-to…

We release the strongest public 1.3B and 3B models so far – the ShearedLLaMA series. Structured pruning from a large model to a small one is far more cost-effective (only 3%!) than pre-training them from scratch! Check out our paper and models at: xiamengzhou.github.io/sheared-llama/ [1/n]

Excited to share that 15 main conference/findings papers by PLI researchers are being presented at EMNLP in Singapore over the next several days! Learn more by checking out the latest PLI Blog post: bit.ly/473Um5H. Feel free to reach out to the authors!

At #EMNLP2023 presenting AutoCompressors with Alex Wettig! Check out the updated paper (arxiv.org/abs/2305.14788) and updated repo (github.com/princeton-nlp/…) where we scale up AutoCompressors to Llama-2 based models. Download our best model at huggingface.co/princeton-nlp/…!

New preprint "Improving Language Understanding from Screenshots" w/ Zirui "Colin" Wang Adithya Bhaskar Danqi Chen. We improve language understanding abilities of screenshot LMs, an emerging family of models that processes everything (including text) via visual inputs arxiv.org/abs/2402.14073

Google's Gemma-7B is good on GSM8K and MATH, but it's underwhelming on our new TutorEval benchmark 🤔 Gemma lags behind Zephyr and Mistral models, even on the short-context TutorEval-ClosedBook task! Results and code: github.com/princeton-nlp/… Paper: arxiv.org/abs/2402.11111

New multimodal benchmark! Very excited to be part of this project showing the huge gap between multimodal LLMs and humans. New models are improving fast but we still have a long way to go! Check out the paper and leaderboard at charxiv.github.io/#leaderboard

🎉Exciting news in Multimodal LLMs!  We're excited to see that 𝐈𝐧𝐭𝐞𝐫𝐧𝐕𝐋 𝐂𝐡𝐚𝐭 𝐕𝟐.𝟎 and 𝐂𝐚𝐦𝐛𝐫𝐢𝐚𝐧 now lead the 𝐂𝐡𝐚𝐫𝐗𝐢𝐯 leaderboard (charxiv.github.io) in chart understanding for open-weight models. 🤔What leads to their success? Here's some of

Really excited to see TutorEval and TutorChat cited in the fantastic LearnLM work! 😀 Come to our #ICML poster session tomorrow to chat about our open-source LLM eval and dataset for science and education 👨‍🎓

🤖 Welcome 𝐆𝐏𝐓-𝟒𝐨 𝐌𝐢𝐧𝐢 and 𝐈𝐧𝐭𝐞𝐫𝐧𝐕𝐋𝟐 𝐋𝐋𝐚𝐌𝐀-𝟑 𝟕𝟔𝐁 to the CharXiv (charxiv.github.io)  leaderboard for chart understanding! As concurrently released models, GPT-4o Mini is 𝐛𝐞𝐚𝐭𝐞𝐧 𝐛𝐲 𝐭𝐡𝐞 𝐨𝐩𝐞𝐧-𝐰𝐞𝐢𝐠𝐡𝐭 𝐨𝐧𝐞. 🎊 Congratulations to

Check out our new retrieval benchmark! We curated a large set of challenging questions about the recent ML literature and evaluated SOTA retrievers, Google, and more! Looking forward to seeing the next generation of retrieval systems support scientific research! 🧑‍🔬