How can we obtain more reliable LLM-based evaluation and better alignment? Introducing Post-hoc Reward Calibration: de-bias your reward model (RM) without extra training or preference data!

Life update 🎉: Today is my first day starting as a postoctoral researcher at UvA Amsterdam ILLC under Ivan Titov!

New paper: "What's New in My Data? Novelty Exploration via Contrastive Generation," w/ Ivan Titov at EdinburghNLP arxiv.org/abs/2410.14765 With no access to the data, but only access to the pre-trained and fine-tuned model, we can reveal novel aspects of the fine-tuning dataset.

My "looking for a postdoc" stickers and I are in Miami for #EMNLP2024! 🤩 Do you have/know of a postdoc for summer/autumn '25 related to interpretability, figlang and/or memorisation (vs generalisation)? Reach out! Looking forward to #GenBench2024 on Saturday and the many many...

We had a blast at this year's ELLIS Schloss Dagstuhl seminar on "Modular and Agentive LLMs". Thanks everyone for participating!

🚨 PhD position alert! 🚨 I'm hiring a fully funded PhD student to work on mechanistic interpretability at UvA Amsterdam. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-Po…

The poster fest I organised was so lively and saw many students showing up to rehearse for the upcoming NeurIPS Conference Thanks #GAIL the #Generative #AI Lab School of Informatics, The University of Edinburgh for sponsoring the event!

Is sparsity the key to conditional computation, interpretability, long context/generation, and more in foundation models? Find out at my #NeurIPS2024 tutorial on Dynamic Sparsity in Machine Learning with Andre Martins! Followed by a panel with Sara Hooker and Alessandro Sordoni 🧵

Two papers got accepted at #ICLR2025 and one at #NAACL2025! One for calibrating the RM bias: openreview.net/forum?id=Iu8Ry… Two for MoE: openreview.net/forum?id=eWNEq…; arxiv.org/abs/2406.18219 Thanks to my great supervisors Ivan Titov Edoardo Ponti and my excellent co-author Zihan Qiu!

Come and join us at AmsterdamNLP! We have two open PhD positions in #NLProc with a focus on multilingual NLP and LLM alignment. Looking for students with an NLP/ML background and an interest in language and society. werkenbij.uva.nl/en/vacancies/t…

I miss Edinburgh and its wonderful people already!! Thanks to Tal Linzen and Edoardo Ponti for inspiring discussions during the viva! I'm now exchanging Arthur's Seat for Mont Royal to join Siva Reddy's wonderful lab Mila - Institut québécois d'IA 🤩

Congratulations Verna! This was one of the best theses I've ever read, I highly recommend checking out Verna's work on the tradeoffs between memorization and generalization in language models! vernadankers.com

Check out the full paper here: arxiv.org/abs/2507.01679… credit to Zeyu Huang, Ivan Titov & other authors.

Finally made it to ICML Conference in gorgeous Vancouver! Presenting work at Actionable Interpretability Workshop ICML2025 on Saturday (more on that soon 👀). If you're into interpretability/RL/AI Safety, I'd love to chat :)

🚨New paper alert!🚨 "Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them" Actionable Interpretability Workshop ICML2025 ICML'25 DeepSeek popularised RLVR and distillation for 'reasoning training'! But how do they differ under the hood? Details in 🧵: (1/8)

Had a fantastic time hosting Wen Lai at EdinburghNLP! The visit led to an elegant light-PEFT method: from just a few examples, it learns sparse, targeted interventions — simple, robust, and easy to use.

🚀 Introducing Prefix-RFT to blend SFT and RFT! SFT can learn more complex problems by mimicking, but can have poor generalization. RFT has better overall performance but is limited by the initial policy. Our method, Prefix-RFT, makes the best of both worlds!

Many thanks to the Actionable Interpretability Workshop ICML2025 organisers for highlighting our work - and congratulations to Pedro, Alex and the other awardees! Sad not to have been there in person, it looked like a fantastic workshop. AmsterdamNLP EdinburghNLP

Congratulations to the winners of the 2025 IJCAI–JAIR Prize for their paper “Compositionality Decomposed: How Do Neural Networks Generalise?” — Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni!   arxiv.org/abs/1908.08351  #IJCAI2025

Proud to accept a 5y outstanding paper award IJCAIconf 🏆 from JAIR for the impact Compositionality Decomposed has had, on behalf of the team w/ Dieuwke Hupkes, elia bruni & Mathijs Mul! 🧡 Come to room 513 on [email protected] to learn about rethinking compgen evaluation in the LLM era 🤖