Are current eval/deployment practices enough for today’s robot policies? Announcing the Eval&Deploy workshop at CoRL 2025 Conference on Robot Learning, where we'll explore eval + deployment in the robot learning lifecycle and how to improve it! eval-deploy.github.io 🗓️ Submissions due Aug 30

I will be in Vienna for #ACL2025! Hit me up if you want to chat about human-AI interaction, building human-centered & reliable LLMs, or just to get coffee. I am also on the job market for postdoc and industry research positions on the above topics (yes, including the coffee part)

College is essentially a walkable city with good infrastructure, full of civilians working toward self-betterment. There's still a respect for expertise and facts, as well as community and forgiveness. Leaving that is tough.

Our paper CTRL-D is accepted to ACL Findings and will be presented at ACL 2025! 🗓️Poster session: 18:00–19:30 (Level 0 Exhibit Halls X4/X5) I’m sad I can’t be there, but Jordan (Jordan Boyd-Graber) will! You’ll enjoy learning about CTRL-D from him. Now… what is CTRL-D? 🔍

I won't be at ACL 2025😢 but Tejas Srinivasan will present this work on Tuesday, July 29 at 4PM in Hall 4/5! Drop by to learn about how we showed that VLMs don't actually understand human actions (<10% acc vs human's 95% acc 🤯) Here's the poster in case you miss it:

Our poster is up! 🎉 We study how factuality evaluation metrics fare in their down-stream use cases uncovering "gotchas" for several NLP subcommunities. I won't be attending in person but Robin Jia has graciously offered to man the post. Longer thread with details coming soon

If you're using AI to write essays, eulogies, a text to your wife, I do think less of you as a person. Ceding your mental and creative abilities to a machine is an embarrassing thing and people should be ashamed to admit doing it in public.

This is how Musk and oligarchs work: Get billions in subsidies from the federal government. Pay little or nothing in taxes. Complain about waste and fraud. Then cut federal programs that working families need. This is why we need a political revolution in this country.

What if survey-derived rubrics 📋 graded ChatGPT instead of vibes? We benchmark LLMs & deep research systems across 75 research fields 🩺🧬🦾⚗️🏛️🎭💹: Perplexity deep research wins > 82% of head-to-heads vs the next best! w/ Li S. Yifei, Chaitanya Malaviya, and Mark Yatskar

I'll be at #COLM2025 from tomorrow!! Hmu if you want to chat about how we can make LLMs reliable collaborators, debate whether using LLMs is rotting our brains, or want to do karaoke 🎤🎶 I'm also organizing the ORIGen workshop on Friday, come through! origen-workshop.github.io/programme

Gross

COLM attendees listening to Nicholas Carlini read excerpts from the nyt "chatgpt teen s**cide" article:

if you're hanging around at the COLM workshops and are interested improving human-LLM collaboration and building reliable+accountable LLM systems, drop by the ORIGen workshop in room 518A tomorrow! origen-workshop.github.io/programme/

Can someone for the life of me explain to me why the whole LLM RL community is researching GSM8K and making general claims from this? It's f*king arithmetic. I kinda got it during NeurIPS, rushed, not much better training data around. But 9 months after R1? Inexcusable.

does this remind you. of anything.

This is going to get me piled on a bit but a video referencing the moon landing and the fall of the Berlin Wall in a post about raising capital for a prediction market encapsulates everything I hate about the tech industry