
Vansh Singh
@vanshcsingh
Finetuning @DbrxMosaicAI. Previously @Stripe.
ID: 934296125695180800
25-11-2017 05:42:09
102 Tweet
218 Takipçi
623 Takip Edilen

Meet DBRX, a new sota open llm from Databricks. It's a 132B MoE with 36B active params trained from scratch on 12T tokens. It sets a new bar on all the standard benchmarks, and - as an MoE - inference is blazingly fast. Simply put, it's the model your data has been waiting for.


Yes, we dropped the best open-source model as of today. Yes, you can train your own SOTA models too. Train on Databricks! databricks.com/blog/introduci…




SnowflakeDB Awesome work training such a big model with a permissive license! I think you had a mistake in your IFEval implementation, your reported number is less than 2x what we observe (though it does vary with inference server and sampling parameters). You should see in the high 60s














