🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

You can just continue pre-train things ✨ Happy to announce the release of BioClinical ModernBERT, a ModernBERT model whose pre-training has been continued on medical data The result: SOTA performance on various medical tasks with long context support and ModernBERT efficiency

Clinical encoders are joining the ModernBERT family ☺️

🚀Announcing BioClinical ModernBERT, a SOTA encoder for healthcare AI, developed by Thomas Sounack Thomas Sounack for Dana-Farber Cancer Institute in collaboration with Harvard University, LightOn, Massachusetts Institute of Technology (MIT), McGill University, Albany Med Health System, Microsoft Research. Seamless continued pre-training enables SOTA

Your daily reminder that fine tuning is just continued pretraining. Super cool results from Antoine Chaffin who is putting this knowledge into practice to improve medical AI:

we are so back "Mitochondria is the powerhouse of the [MASK]."

BioClinical ModernBERT is out! Built on the largest, most diverse biomedical/clinical dataset to date ‼️Delivers SOTA across the board Thrilled to be part of this effort led by Thomas Sounack

codepen.io/jmikedupont2/p… colab.research.google.com/drive/1uSx8yYZ… next demo visualizing BioClinical-ModernBERT-base embeddings on a sphere

BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP → Built on ModernBERT with 8K context, RoPE, and fast unpadded inference Trained via two-phase continued pretraining: - Phase 1: 160.5B tokens (PubMed + PMC + 20 diverse clinical

Exciting to see BioClinical ModernBERT (base) ranked #2 among trending fill-mask models - right after BERT! The large version is currently at #4. Grateful for the interest, and can’t wait to see what projects people apply it to!