‼️ LLMs hallucinate facts even if provided with correct/relevant contexts 💡 We find models' attention weight distribution on input context versus their own generated tokens serves as a strong detector for such hallucinations 🚀 The detector transfers across models/tasks, and can

🤔 In training vision models, what value do AI-generated synthetic images provide compared to the upstream (real) data used in training the generative models in the first place? 💡 We find using "relevant" upstream real data still leads to much stronger results compared to using

Hard negative finetuning can actually HURT compositionality, because it teaches VLMs THAT caption perturbations change meaning, not WHEN they change meaning! 📢 A new benchmark+VLM at #ECCV2024 in The Hard Positive Truth arxiv.org/abs/2409.17958 Cheng-Yu Hsieh Ranjay Krishna uclanlp

I will be presenting our Lookback Lens paper at #EMNLP2024 in Miami! 📆 Nov 13 (Wed) 4:00-5:30 at Tuttle (Oral session: ML for NLP 1) 🔗 arxiv.org/abs/2407.07071 Happy to chat about LLMs and hallucinations! See you soon in Miami! ✈️ Linlu Qiu Cheng-Yu Hsieh Ranjay Krishna Yoon Kim

Introducing AURORA 🌟: Our new training framework to enhance multimodal language models with Perception Tokens; a game-changer for tasks requiring deep visual reasoning like relative depth estimation and object counting. Let’s take a closer look at how it works.🧵[1/8]

(1/5)🚨LLMs can now self-improve to generate better citations✅ 📝We design automatic rewards to assess citation quality 🤖Enable BoN/SimPO w/o external supervision 📈Perform close to “Claude Citations” API w/ only 8B model 📄arxiv.org/abs/2502.09604 🧑‍💻github.com/voidism/SelfCi…

I'm exited to announce that our work (AURORA) got accepted into #CVPR2025🎉! Special thanks to my coauthors: Alan Luo, Cheng-Yu Hsieh, Ethan Shen, Dongping Chen, Linda Shapiro and Ranjay Krishna, This work wouldn’t have been possible without them! See you all in Nashville 🎸!

The 2nd Synthetic Data for Computer Vision workshop at #CVPR2025! We had a wonderful time last year, and we want to build on that success by fostering fresh insights into synthetic data for CV. Join us! We welcome submissions! Please consider submitting your work! (deadline: March

Stop by poster #596 at 10A-1230P tomorrow (Fri 25 April) at #ICLR2025 to hear more about Sigmoid Attention! We just pushed 8 trajectory checkpoints each for two 7B LLMs for Sigmoid Attention and a 1:1 Softmax Attention (trained with a deterministic dataloader for 1T tokens): -

1/8🧵 Thrilled to announce RealEdit (to appear in CVPR 2025)! We introduce a real-world image-editing dataset sourced from Reddit. Along with the training and evaluation datasets, we release our model that achieves SOTA performances on a variety of real-world editing tasks.

Agentic AI will transform every enterprise–but only if agents are trusted experts. The key: Evaluation & tuning on specialized, expert data. I’m excited to announce two new products to support this–Snorkel AI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D! ---

🔥We are excited to present our work Synthetic Visual Genome (SVG) at #CVPR25 tomorrow! 🕸️ Dense scene graph with diverse relationship types. 🎯 Generate scene graphs with SAM segmentation masks! 🔗Project link: bit.ly/4e1uMDm 📍 Poster: #32689, Fri 2-4 PM 👇🧵