A little snapshot of number of pretraining tokens for open source LLMS over the past year. Interesting trend:

Not a surprise but DBRX is already #1 trending on HF!

dbrx-instruct by Databricks Mosaic Research is available on labs.perplexity.ai, enjoy!

Absolutely not. The rules of the game are: If released the model: - You can to compare yourself to other open models as much as you want. If you don’t release the model: - Compare yourself also to GPT-4 before claiming who you outperform or not. Comparing only down: Not fair.

This is literally my new LK-99 🙏🙏🙏

Domain-specific benchmarks matter for enterprise, glad to see DBRX working well. Julia Neagu building some interesting enterprise evals!

DBRX by Databricks ...it's REALLY good!! The New MoE 132b parameter model is open-source and costs $10 m to train. Thank you, Databricks, for your contribution to OS. Check out the full explanation and testing: 🎥👇

"Let's think step by step, using colorful physical analogies..."

🆕 Check out the recent update of 𝕎𝕚𝕝𝕕𝔹𝕖𝕟𝕔𝕙! We have included a few more models including DBRX-Instruct Databricks and StarlingLM-beta (7B) Nexusflow which are both super powerful! DBRX-Instruct is indeed the best open LLM; Starling-LM 7B outperforms a lot of even

we all know how important LLM evaluation is.. 🤔 i’m excited to FINALLY announce that we are starting a new 📢 recipe-based evals team!!! 📢 for our first study, we compared 5 LLM-generated chili recipes with the prompt: “Give me a chili recipe with an interesting twist” (1/n)

Good fast evals are the way. Excited for what Julia Neagu and Freddie Vargus have cooking at Quotient AI !

Friday is LLM battle day. I added DBRX to the financial metrics challenge. Overall, very impressed with DBRX. Main takeaways: • correctly calculated metrics • ranked top 4 fastest models • competitive pricing DBRX was +50% cheaper and +100% faster than models in its tier.

LLM evals are a mess! They are noisy, inconsistent, and contradictory. Scaling laws on the other hand have consistently held up to increasing scrutiny. Can we use the reliability of scaling laws to predict the quality of our eval benchmarks?

me crafting a training yaml at work (it’s going to crash immediately)

Databricks to acquire Tabular (now part of Databricks), a data platform from the original creators of Apache Iceberg. Together, we will bring format compatibility to the lakehouse for Delta Lake and Apache Iceberg databricks.com/blog/databrick…

This Monday in SF, our MosaicX meetup is essentially a mini-conference on AI! Hear from Jonathan Frankle, Sharon Zhou, Jerry Liu, Sarah Catanzaro, Julia Neagu, Yaron, jasmine collins and many others on the latest research.

Super excited about the new Agent Framework, Tool Catalog, Vector Search, Evaluation and Training capabilities we launched today in Mosaic AI. We see more companies building compound AI systems, and we have created an end-to-end environment to do this. databricks.com/blog/mosaic-ai…

Meta AI's Llama release today is really important, likely the most important open source AI announcement ever. Many people don't understand why: 1. The quality gap between the best proprietary and open models has effectively vanished. No one really knew if this gap would get

This is a good opportunity to announce that I recently joined the research team at Databricks where I will be working alongside Jonathan Frankle Rishabh Singh Matei Zaharia Erich Elsen, and many others on the hardest problems at the intersection of information retrieval and AI.

today we're releasing a new small model (0.5B) for detecting problems with tool usage in agents, trained on 50M tokens from publicly available MCP server tools it's great at picking up on tool accuracy issues and outperforms larger models