Huge congratulations and thanks for the gracious shoutout, A. Feder Cooper and team! Our paper was recently accepted to Conference on Language Modeling, so please come check it out if you're attending!

Any paper with Yejin Choi on it is a must read.

cool new work from Jonathan Hayase and friends!

What do BPE tokenizers reveal about their training data?🧐 We develop an attack🗡️ that uncovers the training data mixtures📊 of commercial LLM tokenizers (incl. GPT-4o), using their ordered merge lists! Co-1⃣st Jonathan Hayase arxiv.org/abs/2407.16607 🧵⬇️

a very cool form of training data inference, especially considering the importance of data mixtures (arxiv.org/abs/2305.10429)!

One of my fav new papers!! Always wondered about leakage of tokenizers, checkout this great work!

Tokenizers and autogregressive LMs are both trained to compress text, but tokenizer training is deterministic and we know exactly how it works! This makes inverse problems wrt the data much easier. There's a wealth of info lurking in public tokenizers waiting to be extracted!

📣 After graduating from @UWCSE, I am joining UC Berkeley as an Assistant Professor (affiliated w Berkeley AI Research BerkeleyNLP) and Ai2 as a Research Scientist. I'm looking forward to tackling exciting challenges in NLP & generative AI together with new colleagues! 🐻✨

Join us on August 14th at 3PM Eastern / 12PM Pacific to learn about the three new benchmarks we've recently released: SciCode, AssistantBench and CiteMe. We will also have some SWE-bench updates. The event will be on Zoom. lu.ma/4240w5us

Do you like yellow? Then, according to LLMs, you are probably a school bus driver! Excited to share our new paper about Semantic Leakage in Language Models! Joint work with my wonderful collaborators @terra Alisa Liu luke Noah A. Smith Paper: gonenhila.github.io/files/Semantic… 1/10

data is the secret sauce when cooking LMs; come learn our recipes!🫕 happening now in World Ballroom A on 23rd floor🤩

📢Super excited that our workshop "System 2 Reasoning At Scale" was accepted to #NeurIPS24, Vancouver! 🎉 🎯 how can we equip LMs with reasoning, moving beyond just scaling parameters and data? Organized w. Stanford NLP Group Massachusetts Institute of Technology (MIT) Princeton University Ai2 UW NLP 🗓️ when? Dec 15 2024

OpenAI just released a small subset of SWE-bench tasks, verified by humans to be solvable. I would treat this subset as "SWE-bench Easy"- useful for debugging your system. But eventually when you're ready for launch, we still recommend running on SWE-bench Lite or the full set

👽Have you ever accidentally opened a .jpeg file with a text editor (or a hex editor)? Your language model can learn from these seemingly gibberish bytes and generate images with them! Introducing *JPEG-LM* - an image generator that uses exactly the same architecture as LLMs

Check out JPEG-LM pleaseeeeeee

Check out JPEG-LM, amazing and transformative idea!

Check out JPEG-LM, a fun idea led by Xiaochuang Han -- we generate images simply by training an LM on raw JPEG bytes and show that it outperforms much more complicated VQ models, especially on rare inputs.

Can we train an LM on raw JPEG bytes and generate images with that? Yes we can. Check out JPEG-LM (arxiv.org/abs/2408.08459), a cool work lead by @XiaochuangHano to learn more.

One of my personal most fav lines of work recently!! Work led by Nikita Haduong at Allen School UW NLP with Irene Wang, Roy Lu, and Noah A. Smith Watch out for Nikita, she'll be on the job market soon w/ a v unique set of expertises at the intersection of NLP, Games, Edu, and HCI

🚨 New Paper Alert! 🚨 Introducing 💿 StyleRemix: a novel, interpretable method for authorship obfuscation which perturbs fine-grained *style-elements* in the input text 🎨✍️. 📄 Paper: arxiv.org/abs/2408.15666 🧑‍💻 Code: github.com/jfisher52/Styl… 🚀 Demo: huggingface.co/spaces/hallisk…