1/ Excited to finally tweet about our paper “Implicit meta-learning may lead LLMs to trust more reliable sources”, to appear at ICML 2024. Our results suggest that during training, LLMs better internalize text that appears useful for predicting other text (e.g. seems reliable).

.Bruno Mlodozeniec is giving an AABI contributed talk on Implicitly Bayesian Prediction Rules in Deep Learning openreview.net/forum?id=WBPVW… #ICML2024

Come to our poster tomorrow morning if you're at ICML! Hall C 4-9 #2711 Tue 23 July 11:30 a.m. CEST — 1 p.m. CEST

It’s game night! Playing games, thinking about games, becoming an expert, and inventing entirely new games are key elements of our shared human experience. Yet how do we actually play new games in the first place? And determine what games we even want to play? 1/

It's been great presenting at the #ICML2024 workshops. Thank you to everyone who came up to chat with us! Here are the works that we presented:

Are all blocks in a pretrained LLM equally important? In our new preprint, “A deeper look at depth pruning of LLMs” (arxiv.org/abs/2407.16286), we attempt to better understand the impact of depth pruning, which is a specific case of structured pruning that directly translates to

The biggest take-away for me from this work was that training neural nets with SGD does something akin to implicit model selection. The interesting thing is that, for this to work, the order of the data _really_ matters, unlike in exact Bayesian inference.

We’re extremely excited to announce the NeurIPS Workshop on Bayesian Decision-making and Uncertainty: from probabilistic and spatiotemporal modeling to sequential experiment design! This will take place at NeurIPS 2024, in Vancouver, BC, Canada, either on December 14th or 15th.

[New preprint!] What does it take to build machines that **meet our expectations** and **compliment our limitations**? In this Perspective, we chart out a vision, which engages deeply with computational cognitive science, to design truly human-centric AI “thought partners” 1/

Our agenda paper on alignment and safety of LLMs just got published at TMLR: openreview.net/forum?id=oVTkO… 🥳 The revised version is also now on arxiv arxiv.org/abs/2404.09932.

I made my first ever investment in a startup, Ångström AI They build on ML surrogate models to enable cost-effective simulation of molecular properties (e.g. solubility or binding) in pharmacologically relevant settings. A very hard technical problem that this team might crack.