My new blog post argues from first principles how length normalization in preference learning objectives (e.g., SimPO) can facilitate learning from model-annotated preference data. Check it out! cs.princeton.edu/~smalladi/blog…

Questions for GenAI & copyright researchers (w/ answers in ℂ𝕠𝕋𝕒𝔼𝕧𝕒𝕝: cotaeval.github.io): - Can 𝐬𝐲𝐬𝐭𝐞𝐦 𝐩𝐫𝐨𝐦𝐩𝐭/𝐮𝐧𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠 prevent copyrighted content generation? - Does permissive training data still help if 𝐑𝐀𝐆 fetches copyrighted content?

Can 𝐦𝐚𝐜𝐡𝐢𝐧𝐞 𝐮𝐧𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠 make language models forget their training data? We shows Yes but at the cost of privacy and utility. Current unlearning scales poorly with the size of the data to be forgotten and can’t handle sequential unlearning requests. 🔗:

[𝐒𝐩𝐨𝐭𝐥𝐢𝐠𝐡𝐭 The GenLaw Center '24] Fantastic Copyrighted Beasts and How (Not) to Generate Them. We'll have a spotlight talk at the ICML 24 GenLaw Workshop, and please feel free to reach out and chat more!

Google cannot find the paper you want? Introducing LitSearch, a retrieval benchmark with realistic and challenging scientific literature search questions. Paper: arxiv.org/abs/2407.18940 Data/code: github.com/princeton-nlp/…

Come check out our poster at #ACL2024! I will be at the 4pm poster session, stop by to chat about long-context models

I'm super excited about the *CL workshop we're planning to organize on LLM memorization, & its implications for compliance (privacy/copyright) and capabilities (evaluation/generalization). Plz help by RT and voting in the thread!

It’s great Gavin Newsom signed AB 2013 (by Jacqui Irwin), requiring disclosure of a summary of the data used to build Generative AI systems. There is huge uncertainty about GenAI. Legislation that brings sunlight is great; premature restrictions of dubious soundness are not.

Attending Conference on Language Modeling from 10/6 to 10/9! If you want to chat about GenAI security, privacy, safety, or reasoning (I just started exploring it!), DM me :) & My team at Google AI is looking for interns. Email me ([email protected]) your resume if you are interested.

Very proud to introduce two of our recent long-context works: HELMET (best long-context benchmark imo): shorturl.at/JnBHD ProLong (a cont’d training & SFT recipe + a SoTA 512K 8B model): shorturl.at/XQV7a Here is a story of how we arrived there

Theory + exps in our new work show that preference tuning can move probability mass in unexpected ways, causing aligned models (across scales and settings) to unalign. For example, training a model to prefer "No" over "Never" makes prob of "Yes" increase. arxiv.org/abs/2410.08847

Join us today at 3 pm ET for a discussion on AI safety and alignment with David Krueger 🤩 Submit your questions in advance at the link in the post!

Unlearning allows users request the removal of specific data from a trained model. Sounds great, right? 👿 BUT: we show how adversaries can exploit this to completely DESTROY model accuracy—plummeting to just 3.6% on CIFAR-10 and 0.4% on ImageNet after the attack! (1/n)

Is encouraging LLMs to reason through a task always beneficial?🤔 NO🛑- inspired by when verbal thinking makes humans worse at tasks, we predict when CoT impairs LLMs & find 3 types of failure cases. In one OpenAI o1 preview accuracy drops 36.3% compared to GPT-4o zero-shot!😱

Tune in to our PASS Seminar with Nathan Lambert next Monday! Submit your question via the link in thread :)

Excited for the talk today at 2pm ET! YouTube link here youtube.com/@PrincetonPLI and submit your questions via forms.gle/7GQXAr9aonfvy1… 🤩

Happening today! You can submit your questions for Gillian here: forms.gle/3aoVC7jrG5NYpS…

Congratulations to Ai2 on the exciting Tulu 3 release! We had Nathan Lambert on PASS a few weeks ago to talk all about it. Check out the recording for an easy primer to the paper: youtube.com/watch?v=ltSzUI…

TOMORROW PASS SEMINAR, 12/3 at 1pm ET! Speaker: Percy Liang from Stanford University Live: youtube.com/@PrincetonPLI/… Recordings later at: youtube.com/@PrincetonPLI