Super excited Marin is finally out! Come see what we've been building! Code/platform for training fully reproducible models end-to-end, from data to evals. Plus a new high quality 8B base model, fully documented from start to finish.

🚨Time to push LLMs further! 📚LEXam: The legal reasoning benchmark you’ve been waiting for: • 340 exams, 116 courses (EN/DE) • Long-form, process-focused questions • Expert-level LLM Judges • Rich meta for targeted diagnostics • Contamination-proof, extendable MCQs [1/6]🧵

Check out LEXam, our new Legal Reasoning benchmark! Thanks for the great collaboration Ni Jingwei, Jakob, Etienne, Yang, Yoan, Yinya Huang ✈️ ICLR, Mubashara Akhtar, Florian, Oliver, Daniel,Markus Leippold, Mrinmaya Sachan, Alexander Stremitzer, Christoph Engel, Elliott Ash, and Joël Niklaus!

1/5 How good are AI Model’s Reasoning Abilities? We have created LEXam, a legal reasoning benchmark derived from real law exams available in English and German. ETH Center for Law & Economics Elliott Ash Yoan Hermstrüwer Joël Niklaus OpenAI Google AI DeepSeek Anthropic Grok AI at Meta

🚀 Huge thanks to the community! 📈 LEXam is now the #1 trending evaluation dataset on Hugging Face! Check it out: huggingface.co/datasets?other… 🧠 Built for deep legal reasoning across 340 law exams 👩‍⚖️ Expert LLM judge evaluations, long-form + MCQs #LegalNLP #Benchmark #LLM

🚨New Preprint! In multilingual models, the same meaning can take far more tokens in some languages, penalizing users of underrepresented languages with worse performance and higher API costs. Our Parity-aware BPE algorithm is a step toward addressing this issue: 🧵

Mike.ai launches today! 🚀 Leaving harvey.ai because the legal AI space clearly needs more Suits characters - why should Harvey get all the glory? 🤵‍♂️ Kidding obviously - but I really am leaving. Real news dropping soon! ✌️