Which is better, running a 70B model once, or a 7B model 10 times? The answer might be surprising! Presenting our new Conference on Language Modeling paper: "The Larger the Better? Improved LLM Code-Generation via Budget Reallocation" arxiv.org/abs/2404.00725 1/n

Can you answer these riddles? We are happy to present our new paper “Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Models”. Paper: Website: visual-riddles.github.io 🧵

Exciting news! I'll present my poster at #ACL2024 about unsupervised document structure extraction tomorrow (Aug. 12th) at 12:45 PM 🕒 Come say hi and let's chat over the paper! arxiv.org/pdf/2402.13906 More details below ⬇️ w/ Gabriel Stanovsky (((ل()(ل() 'yoav))))👾 Ai2 HUJI NLP

It would be nice not to worry about catastrophic forgetting while continually training your models, wouldn’t it? Check out our new #ECCV2024 paper to see how you can do that. “MagMax: Leveraging Model Merging for Seamless Continual Learning” 📜arxiv.org/pdf/2407.06322

Human feedback is critical for aligning LLMs, so why don’t we collect it in the open ecosystem?🧐 We (15 orgs) gathered the key issues and next steps. Envisioning a community-driven feedback platform, like Wikipedia alphaxiv.org/abs/2408.16961 🧵

This is the one paper to read if you care about human input into AI systems. You know who else agrees we have to collect and leverage human feedback in open source AI? That's right Yann LeCun and definitely Soumith Chintala

Human feedback is a valuable resource for model development and research 🤖💬 While for-profit companies collect user data through their APIs to improve their own models, the open-source community lags behind What's missing?🧐 We tackle it here⬇️

Human feedback is critical for aligning LLMs, but it’s often locked away in proprietary datasets. In our new paper, we explore scalable methods to collect and share open-source human feedback data for LLM alignment. Let’s democratize this process and push the field forward! 🚀👇

1/ Into Image Captioning? Don’t miss this! Struggling to keep up with the influx of new metrics but still see the same 5 (BLEU, METEOR, ROUGE, CIDEr, SPICE) leading? Read our recent Captioning evaluation survey! arxiv.org/abs/2408.04909 w\ Gabriel Stanovsky Omri Abend Lea Frermann >

🚀 AI Tinkerers & Human Feedback Foundation present Paper Club! Premiere: "Learning from Naturally Occurring Feedback" Sept 17 at 12pm ET. #PaperClub #BuildingAI Shachar Don-Yehiya Leshem (Legend) Choshen 🤖🤗 @ACL Omri Abend Elena Yunusov Human Feedback Foundation - link in first comment

✨ Developed a new benchmark or dataset for language models? ✨ Want the community to trust and adopt it? 🤔 So, demonstrate its validity by comparing it to established benchmarks! BenchBench makes it easy. Check it out: 👉 huggingface.co/spaces/ibm/ben…

Language Models output probabilities for tokens. So probabilities of spans must follow a valid joint, right? Check out our new #EMNLP2024 paper -- "CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models". w/ Yuli Slavutsky and Omri Abend (1/5)

Ever wondered if model merging works at scale? Maybe the benefits wear off for bigger models? Maybe you considered using model merging for post-training of your large model but not sure if it generalizes well? cc: Google AI Google DeepMind UNC NLP 🧵👇 Excited to announce my

1/n Excited to present our work at #EMNLP2024’s Workshop on Narrative Understanding! Testimonies reveal personal stories, making them ideal for studying character development. We use NLP to analyze character arcs in 1,000 Holocaust testimonies. Omri Abend Renana Keydar Eitan Wagner

Look at the CRAZY domain gap we found in summarization datasets: while English resources are diverse, other languages are mostly restricted to news. Presenting our survey following 130+ datasets in 100+ languages! Explore: github.com/edahanoam/Awes… Gabriel Stanovsky, HUJI NLP 1/6

Say I want to compare system qualities - pick between 2 configurations, or rank a whole bunch of models. I'll use LLM-as-a-judge, right? 🧑🏻‍⚖️ But how do I know the LLM judge is up to the task? Who is a good judge for ranking systems? Enter our new paper!✨🧵 arxiv.org/abs/2412.09569

New preprint! ✨ Interested in LLM-as-a-Judge? Want to get the best judge for ranking your system? our new work is just for you: "JuStRank: Benchmarking LLM Judges for System Ranking" 🕺💃 arxiv.org/abs/2412.09569

Happy to share that our paper is now on Arxiv 🤗 🚞 Applying NLP for analyzing character development 👣 ✡️ Examining Religious Trajectories in 1000 Holocaust testimonies 🕎 🍂🌱🍂🌱🌱 Sequence Clustering 🍂🌱🍂🌿🌱 arxiv.org/abs/2412.17063

🔔 New Paper! We propose a challenging new benchmark for LLM judges: Evaluating debate speeches. Are they comparable to humans? Well... it’s debatable. 🤔 noy-sternlicht.github.io/Debatable-Inte… 👇 Here are our findings:

🚨 New Paper: "Time to Talk"! 🕵️ We built an LLM agent that doesn't just decide WHAT to say, but also WHEN to say it! Introducing "Time to Talk" - LLM agents for asynchronous group communication, tested in real Mafia games with human players. 🌐niveck.github.io/Time-to-Talk 🧵1/7