Beyond their use in assisting human evaluation (e.g. CriticGPT), can critiques directly enhance preference learning? During my @Cohere internship, we explored using synthetic critiques from large language models to improve reward models. 📑Preprint: arxiv.org/abs/2405.20850

I’ll be at ICML next week presenting our work on efficient initialisation for mixture of experts (arXiv link dropping soon) Come to our spotlight talk & poster sessions and say hi! 💙

Excited to share The AI Scientist! We use LLMs to autonomously come up with research ideas, implement them, do literature search, write them up, and review them -- producing full-length papers on AI without human intervention. Co-led with Cong Lu and Robert Lange

I'm incredibly proud that Aya received #ACL2024 Best Paper Award 🥹. Huge congratulations to the Aya team and Cohere For AI community who make this possible by for extending frontiers of LLMs to multilingual, building Aya Model and Aya Dataset 🌿🌏

Checkout below our new work on adaptable MoEs led by Nikolas Gritsch! adaptability, specialization, and efficiency all in one framework☝️

How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this: Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢 🧵⬇️

I'll be at #NeurIPS2024 this week with 2 works on pretraining MoEs: 1) BAM: Why & how to use attention experts for MoE upcycling (Wed 4:30 pm @ East Exhibit Hall A-C #3011) x.com/IreneZhang30/s… 2) Nexus: An MoE framework that easily adapts to new data distribution, led by the

I'm hiring performance engineers for the pre-training team at Cohere. If you enjoy writing efficient kernels, hardware-aligned architecture design and optimisations, do reach out! Check out the live job posting here: jobs.ashbyhq.com/cohere/d42f5fd…