🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Happy to share LightSeq is accepted by Conference on Language Modeling 🥳 LightSeq supports efficient long-context Transformer training, where the supported context length grows with the number of nodes. We are excited about the innovative applications it will enable, such as long-context LLM/VLM! 🚀

When talking abt personal data people share w/ OpenAI & privacy implications, I get the 'come on! people don't share that w/ ChatGPT!🫷' In our Conference on Language Modeling paper, we study disclosures, and find many concerning⚠️ cases of sensitive information sharing: tinyurl.com/ChatGPT-person…

🔥We release the first open-source 1.4T-token RAG datastore and present a scaling study for RAG on perplexity and downstream tasks! We show LM+RAG scales better than LM alone, with better performance for the same training compute (pretraining+indexing) retrievalscaling.github.io 🧵

Introducing Molmo: a family of open state-of-the-art multimodal language models that push the frontier of vision by pointing to reason about images! The data curation has been excellent and is leading to our ridiculously tiny 7B models to outperform on par or surpass top models

Excited to be at #NeurIPS2024! Please reach out if you want to chat about synthetic data, inference time algos, or anything else :) I'll present our paper measuring the true utility of synthetic training images. Come check us out! 🕓 Thurs 4:30pm 📍East Exhibit Hall A-C #1602

Excited to present MediQ at #NeurIPS ! 📍Stop by my poster: East Exhibit Hall A-C #4805📷 🕚Thu, Dec 12 | 11am–2pm 🗓️tinyurl.com/mediq2024 Love to chat about anything--reasoning, synthetic data, multi-agent interaction, multilingual nlp! Message me if you want to chat☕️🍵🧋

Hanna Hajishirzi and I are looking for postdocs/students to work on AI for science, including foundation models for scientific literature (as in openscholar.allen.ai) + scientific data (genomics, images, molecules, ...). Let us know if interested and please help RT! 🧪

he chonky huggingface.co/allenai/Llama-…

Want to quickly sample high-quality images from diffusion models, but can’t afford the time or compute to distill them? Introducing S4S, or Solving for the Solver, which learns the coefficients and discretization steps for a DM solver to improve few-NFE generation. Thread 👇 1/

Is a single accuracy number all we can get from model evals?🤔 🚨Does NOT tell where the model fails 🚨Does NOT tell how to improve it Introducing EvalTree🌳 🔍identifying LM weaknesses in natural language 🚀weaknesses serve as actionable guidance (paper&demo 🔗in🧵) [1/n]

Think PII scrubbing ensures privacy? 🤔Think again‼️ In our paper, for the first time on unstructured text, we show that you can re-identify over 70% of private information *after* scrubbing! It’s time to move beyond surface-level anonymization. #Privacy #NLProc 🔗🧵

The craziest paper I've been on for a while. Qwen 2.5 Math w RLVR can learn with random rewards per rollout prompt in GRPO due to some funky clipping of the logratios and qwen's earlier training heavily on code-integrated reasoning for math.

haha random rewards go brrr fun work co-led with Rulin Shao Stella Li Rui Xin tldr, you can just, like, do things :0

One more fun thing! RLVR can elicit existing behaviors like code reasoning. But! If your model is not good at code but thought it could? - RLVR w/ spurious rewards let Olmo use more code: but perf decreased (Fig 6) - When we discourage it not to: the perf goes up!🤣 (Fig 9)