Not a problem with Lakehouse.

Tutorial for the soft brush workflow is out now! Super useful for quickly compositing the best parts of different generations into a single final image.✨

Function calling significantly enhances the utility of LLMs in real-world applications; however, evaluating and improving this capability isn't easy — and no one benchmark tells the whole story. Learn more about our approach in the latest blog from Databricks:

Tool-use is definitely key to taking LLMs to the next level - Compound AI Systems. But measuring a model's ability to do it is surprisingly tricky. Implementing different evals and analyzing model behavior even on just function-calling gave us a lot of insights!

Would you share your friends’ emails for free pizza? A bunch of students did. Regardless of whether it means that people truly care about privacy, Jonathan Frankle thinks studies like this allow policymakers to make good decisions. mrkup.org/uAATP

It was amazing to reconnect with Ross Teixeira at The Markup to talk about tech policy and the great data intelligence work we're doing at Databricks. I feel very lucky to work at a place where what's good for business is the same as what's good for the world.

My new take on LoRA Learns Less & Forgets Less: 1. Authors added gate_proj, used a=2*rank from prev suggestions. Prev gap on LoRA doing worse shrinks 2. LoRA does well on diverse datasets like Tulu 3. Lots of nuggets in openreview.net/pdf?id=aloEru2… - great work Dan Biderman & team!

Some personal news: I'm thrilled to have joined @Databricks Databricks Mosaic Research as a Research Scientist last month, before I start as MIT faculty in July 2025! Expect increased investment into the open-source DSPy community, new research, & strong emphasis on production concerns 🧵.

Questions about the #DSPy roadmap? Meet the authors at the South Bay GenAI meetup in Mountain View this Thursday! Omar Khattab and Arnav Singhvi of Databricks, Krista Opsahl-Ong and Dilara Soylu of Stanford NLP Group, will be presenting. RSVP: meetup.com/gen-ai/events/…

Personal News: I started today at @Databricks Databricks Mosaic Research as a Research Scientist Intern in the Bay area! 🧑🏻‍💻 ✨ I'm excited to push research forward in the RAG team, build standardized benchmarks, and improve RAG-based evaluations. Anyone in SF, let's hangout and meet up! 🤝

I promise I've done other things since 2018!!! That said, neat looking paper, and cool to see my most recent work with Dan Biderman (first author) connected to my most notorious work. Just wish Databricks Slack wasn't so active every time "lottery ticket" is in a paper title.

Dear Pruning Twitter, What libraries are you using to do structured pruning on LLMs these days? Inquiring students want to know. Regrettably, Jonathan

Excited to announce our new work: Critique-out-Loud (CLoud) reward models. CLoud reward models first produce a chain of thought critique of the input before predicting a scalar reward, allowing reward models to reason explicitly instead of implicitly! arxiv.org/abs/2408.11791

LLM as a judge works well by burning extra Inference compute on chain of thought and self critiques. Reward models work well due to Bradley Terry style objectives being a good fit for most current preference datasets Now you can have the best of both worlds!

Reward models that generate before scoring (similar to LLM as a judge) Open research is slowly chugging through all the reward modeling "tricks" I heard about in top labs recently. Great progress in this area of post training in 2024.

Super excited to join Databricks Mosaic Research team Databricks Mosaic Research! Looking forward to solving important research challenges in foundation models with my amazing team members and unlocking their full potential to accurately solve real-world problems.

I promise I didn't study this in the 90s.

Huge congrats to Dr. Sara Hooker for defending her PhD! Sword will be in the mail shortly ⚔️

The Economist just published a deeply-researched story about car bloat, and it's very, very damning: "For every life that the heaviest 1% of SUVs and trucks save, there are more than a dozen lives lost in other vehicles." Well worth your time: economist.com/interactive/un…

BREAKING: we have achieved AGI internally (trained a bufo diffusion model). Taking requests for custom bufo images