So you've created an ✨awesome✨ biomedical ML model, and now you want to (responsibly) share it with the world Here are some best practices for sharing medical models and demos ⤵️

You know something is trustworthy when adding the word “trustworthy” in front of it doesn’t make any sense. Like is anyone out there doing Trustworthy ANOVA research?

Foundation models (FMs) such as #ChatGPT have the potential to revolutionize healthcare. But what's hype and what's real? This review from a team in Stanford Medicine includes 84 clinical FMs + proposes an evaluation framework better suited to assess value. nature.com/articles/s4174…

Our survey on #LLMs for EHRs is out! We review 84 published Foundation Models for EHRs, discuss current limitations and opportunities. Great work by Michael Wornow et al! 📝 Blog post hai.stanford.edu/news/shaky-fou… Stanford HAI 🎓 Manuscript nature.com/articles/s4174…

The hypocrisy of starting the Doerr School of Sustainability and then deciding to ship athletic teams across the country every weekend by plane is astounding

Lots of hype around #LLMs in healthcare. What do clinicians really want from an #LLM? We asked them! Introducing #MedAlign, the first dataset of clinician-generated instructions + responses for EHRs 🏥🤖 📄Paper: arxiv.org/abs/2308.14089 🌐Website: medalign.stanford.edu

Super excited to share our benchmark dataset + eval for #LLMs on Electronic Health Record (EHR)-based tasks! If you're curious how clinicians 🩺 will likely use LLMs in coming years, see Jason Alan Fries' 🧵👇 + check out our 📄 (arxiv.org/abs/2308.14089)

🩺 Clinicians plz help! Large Language Models (#LLMs) can do a lot, but what do you _want_ them to do? Have an instruction you wish an LLM could handle at the bedside? Submit here: bit.ly/medalign to help us build LLMs aligned with clinician needs and preferences 🙏

More meaningful evaluation of medical LLMs is a key challenge currently, so it's really exciting to see work like this 🤩 You can even contribute your ideas for tasks involving electronic health records (link on their website)

Great resource from Katie Link! Super excited and encouraged by the love and attention ❤️ that Healthcare 🩺 is getting from the AI community 🤖

On leaderboard Stable LM 3b matches LLaMA v2 7b performance on 42% of the size (beats on SciQ, MMLU), similar architecture, runs much faster. Beats Falcon 7b, MPT-7b etc. It beats all 3b models, including fine-tuned ones. Smol, open LMs ftw 😍 huggingface.co/spaces/Hugging…

Can we build AI research agents to perform long-horizon tasks like ML engineering tasks e.g.Kaggle? Introducing our new work MLAgentBench: Benchmarking Large Language Models as AI Research Agents!

Huge thanks to Sehj Kashyap for highlighting our MedAlign work in this concise &!informative video: youtu.be/e-3TCn8WTMc?si… Come find us at #ML4H ML4H in New Orleans today (Dec 10) where we’ll be giving a ⚡️talk at 15:35 CT! (And Follow/Subscribe Sehj Kashyap!)

Michael is not only incredibly smart, talented, and hard working, but also just a genuinely good person who cares deeply about science, rigor, and other people’s wellbeing. Couldn’t recommend this opportunity highly enough!

Evaluating few-shot learning is standard w/ LLMs but not EHR foundation models... yet! We're excited to release #EHRSHOT a dataset of ~7k patients + a foundation model pretrained on 2.57M de-identified EHRs #NeurIPS2023 📄 arxiv.org/abs/2307.02028 🌐 ehrshot.stanford.edu

Super excited to have led this work with my awesome collaborators! Check out our poster #440 at #NeurIPS2023🗓️ on Thursday from 5-7pm CST to learn more. 📽️ Slides/Talk: nips.cc/virtual/2023/p… 🌐Website: ehrshot.stanford.edu

We're excited to introduce #INSPECT a large-scale ✨3D multimodal✨medical imaging dataset #NeurIPS2023 19,402 Stanford Medicine Patients 🩻23,248 CT Scans + 📄Paired Radiology Notes 📈Longitudinal EHRs 🩺Clinician-validated task labels #DataCentricAI #Multimodal #3Dimaging 1/

Super excited to present our work on MedAlign AAAI #AAAI24 in the AI for Social Impact Track. Happening now in Room 217, come check out our work and the other great projects being presented if you're attending

Are #LLMs ready for deployment into the clinic? How can we tell if they are vs. are not? Jason Alan Fries does a great job laying out the current state of affairs for evaluating medical LLMs and how our recent work, MedAlign (medalign.stanford.edu), fits into the bigger picture.

Our clinical #NLP work just published in Nature Medicine! We present a framework to adapt & evaluate #LLMs for summarization. Physicians 🩺 prefer #LLM summaries to those of #medical experts❗ Big step to reduce documentation 📚 and focus more on personalized care 🙌 A 🧵