SAE Made Easy github.com/EvolvingLMMs-L… Sparse Autoencoders (SAE) have become a cornerstone in the field of explainable AI, powering safety and interpretability research at leading labs like Anthropic, OpenAI, and Google. Despite their effectiveness, training SAEs has

😻 LMMs-Eval upgrades to v0.4, better evals for better models. - multi-node evals, tp+dp parallel. - new doc_to_message support for interleaved modalities inputs, fully compatible with OpenAI official message format, suitable for evaluation in more complicated tasks. - unified

Throughout my journey in developing multimodal models, I’ve always wanted a framework that lets me plug & play modality encoders/decoders on top of an auto-regressive LLM. I want to prototype fast, try new architectures, and have my demo files scale effortlessly — with full

🚀 Releasing LMMs Engine by EvolvingLMMs‑Lab — a lean, flexible framework for any-to-any modality pretraining & fine-tuning. 🔧 Built with cutting-edge optimizations: FSDP2, Ulysses Sequence Parallel, Flash Attention 2 📚 Dive in: github.com/EvolvingLMMs-L…

🔥One-Stop Training Engine for Unified Models🔥 ⚡️LMMs-Engine⚡️ is a lean and flexible unified model training engine built for hacking at scale * Support multimodal inputs and outputs, from AR, diffusion and linear models, to unified models like BAGEL 🏠github.com/EvolvingLMMs-L…

OpenMMReasoner Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe

Love to see more fully open post-training recipes (this one multimodal reasoning). It's surprising how rare post-training data is because the opportunity for impact is huge. Lots of people will try it and simple data methods still can improve on SOTA.

LongVT Incentivizing "Thinking with Long Videos" via Native Tool Calling

🔥 Introducing LongVT: Teaching Multimodal LLMs to "Actively Look Back" and understand long videos just like humans! We tackle the "sparse evidence" & "hallucination" issues in long-video reasoning with an end-to-end Agentic solution. Project: evolvinglmms-lab.github.io/LongVT/ Paper:

Our open-source tools and models have become trusted infrastructure for the global AI community, with representative repos including: 🚂 LMMs-Engine: github.com/EvolvingLMMs-L… 📊 LMMs-Eval: github.com/EvolvingLMMs-L… 🤖 LLaVA-OneVision-1.5: github.com/EvolvingLMMs-L…