How does Google optimize its research and systems? We’ve revealed the secrets behind the Vizier Gaussian Process Bandit algorithm, the black-box optimizer that’s been run millions of times! Paper: arxiv.org/abs/2408.11527 Code: github.com/google/vizier Compared to other industry

Very nice work on amortizing over model components (priors, likelihoods) in addition to the traditional amortization over model parameters in simulator-based inference, with a very useful application to sensitivity analysis.

it's fascinating watching Ted Chiang (my favorite SF author) deeply misunderstand the most impactful technological development of my adult life it's not just that I disagree with him, but that he clearly didn't research it at all, and says provably untrue things

Introducing Project Sid: the first simulations of 1000+ truly autonomous agents collaborating in a virtual world, w/ emergent economy, culture, religion, and government Humans are the only species to land the moon, because we can cooperate at a vast scale Can AI do the same?

Computers have become too hard to use. We found that intelligence (IQ) is a significant predictor of people's ability to complete everyday tasks like inserting page numbers to a Word doc. This contrasts the view that success is just about experience. And that's bad news: 🧵

About to start, pretty excited! Our new seminar course on Generative AI at Computer Science @ University of Helsinki. We focus mostly on LLMs, and a sprinkle of diffusion models. It'll be fun! Co-lecturing with N. Loka & Paul

METR's external evaluation of GPT-4o is pretty interesting stuff: "We reviewed around 150 of the GPT-4o agent’s failures and classified them... around half of them seem plausibly fixable in task-agnostic ways (e.g. with post-training or scaffolding)." metr.github.io/autonomy-evals…

When you study statistics, people just start talking at you like you're supposed to understand the difference between "probability" and "likelihood." If you're confused, you're not alone: these words have an *identical* meaning in English—but an important distinction in math.

New work with a 🌟 team of collaborators! Can we make probabilistic inference FAST using modern deep-learning *amortized* methods, while retaining accuracy guarantees of MCMC? Read on to find out... (Out-take titles: - The best of both worlds - Fast and rigorous & others...)

For only $143, Elsevier/Springer will send me a PDF of my accepted paper that I can share with 25 people :). Publishers again adding tremendous value. I wouldn't know what to do without this attractive option.

Wondering how GPT and Llama compare under the hood? I built a step-by-step code notebook to break down the key differences: github.com/rasbt/LLMs-fro…

Our paper on consistency models for simulation-based inference has been accepted at #NeurIPS2024! Thanks to a fantastic team Valentin Pratz (co-lead), Ullrich Köthe, Paul Bürkner, Stefan Radev 📄 Preprint: arxiv.org/abs/2312.05440 🔁 Updates, code, and thread follow soon™

great opportunity to do a #PhD in #Europe in #ML #AI 🚨🚨🚨 I'll hire 2 students via #ELLIS this year, reach out if you want to do research in: - #reliable and #efficient ML in the wild - scalable #neurosymbolic #nesy AI - #lowrank representations - #tractable inference 🚨🚨🚨

This was a very long road - I did not believe the stories about length of stats publication processes when I started doing research. But even with delays this paper seems still timely: How to do meta-analysis based on reported posterior distributions.

recent *review paper* on (neural) amortised simulation-based inference. I haven't seen it publicized much here among my connections, so there you go

A common misconception about Transformers is to believe that they're a sequence-processing architecture. They're not. They're a *set-processing* architecture. Transformers are 100% order-agnostic (which was the big innovation compared to RNNs, back in late 2016 -- you compute

We present a simple and effective method to reduce the cost of simulations in SBI. The source code is available at github.com/huangdaolang/c…