🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics

Cool stuff! The full MLE-Bench was nearly impossible for anyone outside the big players to run. This will benefit the community so much although it might look like a simple update And, Weco AI's AIDE still performs best on this lite version of the benchmark!

Neat! I haven't looked too closely but this seems thoughtfully done and probably very useful to many people. :) (Similar in spirit to our release of MLE-bench Lite yesterday too -- tis the season for minifying!)

i find mishearing 'agents' as 'asians' way more entertaining "a swarm of asians working 24/7 while you sleep" "the future is asians" "you'll be able to deploy millions of asians in parallel"

New paper: We train LLMs on a particular behavior, e.g. always choosing risky options in economic decisions. They can *describe* their new behavior, despite no explicit mentions in the training data. So LLMs have a form of intuitive self-awareness 🧵

Accepted to ICLR :)

We find that across the board, today’s top LLMs still make genuine mistakes on these benchmarks. At the same time, on the majority of the *original* benchmarks, over 50% of “model errors” are actually caused by label noise! 3/5

🧵 Announcing Open Philanthropy's Technical AI Safety RFP! We're seeking proposals across 21 research areas to help make AI systems more trustworthy, rule-following, and aligned, even as they become more capable.

AIME I 2025: A Cautionary Tale About Math Benchmarks and Data Contamination AIME 2025 part I was conducted yesterday, and the scores of some language models are available here: matharena.ai thanks to Mislav Balunović, Nikola Jovanović et al. I have to say I was impressed,

Offline features like battery level

It's hard to live like short ASI timelines are true, even if you're deeply intellectually convinced. It means going against all of the career advice and norms you've absorbed and rethinking your entire lifecycle, creating a lot of emotional tension to resolve.

This quick experiment I just did made my jaw drop... You can literally create and play any game by iterating over images with the new Gemini model! 🤯

To interpret AI benchmarks, we need to look at the data. Top-level numbers don't mean what you think: there may be broken tasks, unexpected behaviors, or near-misses. We're introducing Docent to accelerate analysis of AI agent transcripts. It can spot surprises in seconds. 🧵👇

We are at ICLR, come chat!