We are thrilled that Isabel Papadimitriou (Isabel Papadimitriou) will be joining UBC Linguistics as an Assistant Professor as of Sept 2025!

Say hi to Eric Zelikman ✈️ ICLR at #ICLR2024 who is presenting our work today on the state of using Vision-Language Models for image description evaluation! Read the paper here: openreview.net/pdf?id=j0ZvKSN…

I submitted a paper to the Journal of Linguistics, and I received one of the most insightful and valuable reviews of my entire career. It included an ingenious new experimental idea that worked out beautifully. If you are out there, dear anonymous reviewer – thank you so much!

Sometime in the next few months, Anthropic is expected to release a research report/paper on sparse autoencoders. Before this happens, I want to make some predictions about what it will accomplish.

Overall, I think that the Anthropic SAE paper, when it comes out, will…

At ICLR? Don't miss the DSPy spotlight poster on Wednesday 4:30 PM (GMT+2).

The DSPy team at ICLR will be represented by Keshav Santhanam and Krista Opsahl-Ong.

I won't be there in person but I might join on an iPad at the session! LMK if you'd want to e-meet!

And the paper Omar Khattab has been waiting for -- we indexed ClueWeb09 and NeuCLIR Track 🔎 TREC 2024 1 with a hierarchical indexing scheme.
We use it to simulate the case where a firehose of docs is coming in and you want to search them.
arxiv.org/abs/2405.00975
4/?

The creator of DSPy, Omar Khattab, talks about its past, present, and the future. How it works and where we should focus next.

❊Very Few Parameter Fine tuning w/ ReFT and LoRA❊

And thanks for great work by Zhengxuan Wu of Stanford NLP Group /
Stanford AI Lab on the ReFT library!

TIMESTAMPS:
0:00 ReFT and LoRA Fine-tuning with few parameters
0:42 Video Overview
1:59 Transformer Architecture Review…

oh one thing i'm looking forward to about presenting ReFT is tricking engineers (who will think we're just interested in benchmark hill-climbing) into learning about interpretability

Oh, look, I hear there's a DSPy meetup in San Francisco on Wednesday (May 1st) from 5:30 PM - 8:30 PM with arize-phoenix, cohere, Weaviate • vector database.

See you there, everyone!

Another class I'm teaching this semester is 'Programming w/ LLMs'. This sidesteps the whole chatbot / assistant / 'an AI' theme and looks at LLMs as function approximators -- where, weirdly, the function needs to be 'found' first.
(Yes, DSPy will feature heavily.)

We are thrilled to announce the inaugural Compound AI Systems Workshop.

sites.google.com/view/compound-…

The event will be hosted on June 13th in the Moscone Center, co-located with the Data + AI Summit.

It will feature sessions with our invited speakers and organizers, including…

a mini-update on ReFT hyperparameter tuning!

we reran LoReFT, using 3 epochs and a batch size of 16 — the same settings as in DoRA/LoRA. ReFTs remains SoTA, and logs (3 seeds) are attached!

w&b logs: wandb.ai/wuzhengx/ReFT_…
code (try it): github.com/stanfordnlp/py…

A striking analysis! A high-level takeaway: just as with essentially every other area of AI, optimizing prompts can create solutions that are highly effective and unlikely to be found with manual exploration.

The picture of Atticus in this announcement captures him so well, but these days he has an amazing beard: maverickphilosopher.typepad.com/.a/6a010535ce1…

This week at CARE (Thursday 11am EST) we have Atticus Geiger presenting his fantastic line of work (with Christopher Potts, Thomas Icard, noahdgoodman, and others) on finding causal abstractions of large language models. Details here: portal.valencelabs.com/events/post/un…

After a meteoric rise, DSPy is now the Stanford NLP Group repository with the most GitHub stars. Big congratulations to Omar Khattab and his “team”.

DSPy: Programming—not prompting—Foundation Models
github.com/stanfordnlp/ds…

New paper! 🫡

We introduce Representation Finetuning (ReFT), a framework for powerful, efficient, and interpretable finetuning of LMs by learning interventions on representations. We match/surpass PEFTs on commonsense, math, instruct-tuning, and NLU with 10–50× fewer parameters.

thanks AK for sharing.

want to mention that, although ReFT is like a ML technique that hill-climbs, interpretability insight (esp. linear subspace) plays a significant role.

e.g., ReFT subspaces are composable (appendix G).

ReFT: Representation Finetuning for Language Models

10x-50x more parameter-efficient than prior state-of-the-art parameter-efficient fine-tuning methods

repo: github.com/stanfordnlp/py…
abs: arxiv.org/abs/2404.03592