Excited to share our newest work! 📝 Evaluation of LLMs is hard, especially for health equity. We provide a multifaceted human assessment framework, 7 newly-released adversarial datasets, and perform the largest human eval study on this topic to date. 🧵: arxiv.org/abs/2403.12025

Landed in Puerto Rico for #SAIL2024

As our attendees settle in the Hyatt Regency Grand Reserve Puerto Rico, we get ready for a deep dive into medical AI with Tristan Naumann and Stephen Pfohl presenting “Red Teaming to Test Limitations of LLMs”. Welcome everyone to #SAIL24! We hope you enjoy the conference and beautiful Puerto Rico. 🏝️

Red teaming LLM’s w Tristan Naumann Microsoft Research Stephen Pfohl Google Research #SAIL2024 #AI detecting threats across the broad taxonomy of LLM vulnerabilities

Made it to Rio for #FAccT2024

Really great to see this published! So, what is this all about? 🧵 (1/