We've open-sourced the code and models for Self-Play Preference Optimization (SPPO)! 🚀🚀🚀 ⭐ code: github.com/uclaml/SPPO 🤗models: huggingface.co/collections/UC…

1/8 ⭐General Preference Modeling with Preference Representations for Aligning Language Models⭐ arxiv.org/abs/2410.02197 As Huggingface Daily Papers: huggingface.co/papers/2410.02… We just dropped our latest research on General Preference Modeling (GPM)! 🚀

1/n 🚀 Introducing General Preference representation Model (GPM) and General Preference Optimization (GPO) for RLHF! 🎯 Reward modeling plays a central role in RLHF. Most existing reward models are based on the classical Bradley-Terry (BT) reward model. However, the BT model has

Excited to see Meta's new paper! Been pondering this exact idea for the past 6 months but never had the resources to verify it properly - and now it's confirmed! Here are some thoughts and ideas I've been exploring that weren't covered in the paper: The key insight, from my

Why are there synchronized ups and downs for chosen and rejected log-probs during DPO (and *POs: IPO, SimPO, CPO, R-DPO, DPOP, RRHF, SlicHF) training? Why do chosen logps decrease, and rejected logps sometimes increase? Our answer: Gradient Entanglement! arxiv.org/abs/2410.13828

Want to take your LLM inference scaling to the next level? Our ICLR paper explores how optimization methods such as ACO and multi-start initialization can help enhance LLM generations. Check it out! 📜 Arxiv: arxiv.org/abs/2411.05010 🖇️ Website: codespace-optimization.github.io 🧵1/n

We’re rolling out Deep Research to Plus users today! Deep Research was the biggest “Feel The AGI” moment I’ve ever had since ChatGPT. And I’m glad more people will experience their first AGI moment! The team also worked super hard to make more tools including image citations /

Excited to present with Isa Fulford tonight at the OpenAI Forum, introducing the research behind Deep Research! Join us at 6pm PT to explore how this new agentic capability in ChatGPT works. Register here:

Just tested Llama4-Scout on our MATH-Perturb benchmark. There is a surprising 18% gap between Original and MATH-P-Simple, making it unique among the 20+ models that came out after 2024. 😂😂 🔗Leaderboard available at math-perturb.github.io. x.com/KaixuanHuang1/…

Grok 3 is now available on Microsoft Azure