Felt cute. Did some petabyte scale preprocessing. Might delete later.

LLM evals are a mess! They are noisy, inconsistent, and contradictory. Scaling laws on the other hand have consistently held up to increasing scrutiny. Can we use the reliability of scaling laws to predict the quality of our eval benchmarks?

1/ 2023 was the year enterprise AI hit an inflection point. This month, I peer into my crystal ball to chart where the field is heading in 2024 & 2025. Founders, take note - the future is coming faster than you think!

When you invent a category, you expect to be the leader. We're really pleased to see that Forrester recognizes our commitment to excellence as THE leader in their new #ForresterWave for Data Lakehouses.
sprou.tt/1eh2T9qU2jh

Super cool that this state-of-the-art medical NLP result was achieved using Omar Khattab's #DSPy . Beginning of the age of auto-optimized compound AI systems?

IReRa is another example of an auto-optimized SOTA pipeline (arxiv.org/abs/2401.12178).

🔥Congratulations to Augustin Toma et al (@BoWang87 Lab) for winning the 1st place in *every* sub-task of the 2024 MEDIQA Clinical NLP competitions.

They build & optimize extremely high-quality LM programs in DSPy, outperforming the next best participating system by up to 18 pt!

🆕💡🎧 DBRX Unpacked - featuring Hagay Lupesko @Databricks:
🔥 Open LLM bridging quality & cost for AI apps
⚡️ Mixture-of-experts architecture for efficiency
⚙️ Optimized for enterprise needs: code gen, long context
thedataexchange.media/dbrx-a-leap-fo…

Weekly LLM drops are cool, but real solutions are systems. Check out Matei Zaharia workshop on it!

Databricks Asset Bundles (DABs) is now GA. It lets you deploy data and AI projects as config-based packages and integrates great with Git, CI/CD and IDEs -- greatly streamlines data and AI deployment using modern software engineering practices. sprou.tt/1ubhaEia3Eu

Come see Wei-Lin Chiang lmsys.org Charles Packer @memgpt Abhi Venigalla talk about LLMs at this month's Berkeley LLM meetup -- hosted Databricks SF along with Andy Konwinski!

lu.ma/berkeleyllm

I don’t think everyone has comprehended the massive disruption and distortion that is going to happen in the Gen AI market due to Llama3. Moats will be destroyed and investments will go to zero.

Just like everything in Gen AI, this will all happen fast.

Awesome release, congrats AI at Meta, and excited that Databricks is a launch partner!

Another major data provider on Databricks Marketplace — welcome ICE!

In our upcoming webinar, join Reynold Xin and Narinder Singh of Salesforce to learn how an AI-optimized data warehouse delivers better performance, governance, and user experience 👇
dbricks.co/49LTHqx

Check out the details of how Allen Institute for AI OLMo 1.7-7B trained on Databricks jumped in accuracy since the last release - spoiler alert 🚨 data matters

Super cool work, so excited that we got to support it!

My GTC talk on LLM inference performance is available on demand now. Watch it to learn about important factors affect LLM inference performance as well as how to think about LLM inference services.