🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Tagging Maximilian Holsman who led this great work!

Happy to share my first paper at Apple, led by Roy Xie. TL; DR: Interleaving <think> and <answer> blocks during reasoning reduces the time-to-first-token *and* improves accuracy.

Looking forward to visiting Stanford this Thursday! Check out my talk at the NLP seminar if you’re around :)

🚀 Introducing Mixture-of-Agents Alignment (MoAA), a new method to "distill" the collective intelligence of open-source LLMs into a single, efficient model. MoAA outperforms GPT-4o as a teacher, boosting smaller models like Llama3.1-8B to rival models 10x their size!

Backtracking allows reasoning models to go back and correct mistakes in their solution attempts. What sorts of tasks benefit from this behavior? And can we boost it using SFT? Hongyi James Cai 's new preprint answers these questions and more -- check it out!

Want to train LLMs with less cost? We introduce BiClip, a clipping-based method that `approximates' adaptive optimizers without maintaining expensive preconditioners

At WWDC we introduce a new generation of LLMs developed to enhance the Apple Intelligence features. We also introduce the new Foundation Models framework, which gives app developers direct access to the on-device foundation language model. machinelearning.apple.com/research/apple…

The technical report for the second generation of Apple Foundation Models is out.. Its been a great year contributing to this effort and being part of an amazing team!