Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive? Identifies a factor that induces unpredictability in multiple-choice assessments of frontier AI models arxiv.org/abs/2406.04391

Really grateful to have worked with this amazing team of researchers to better understand the landscape of predictability in the downstream capabilities of scaling AI systems!

I sadly couldn't attend #ICML2024 but our paper received ✨Outstanding Paper ✨ at the TiFA Workshop ICML workshop! 📈🤔 Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive? 📈🤔 Why should you care? 1/N