🚨 Introducing Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluation. Given an image pair, it is easier for an MLLM to identify fine-grained visual differences during VQA evaluation than to independently detect and describe such differences 🧵(1/n):

Can RL fine-tuning endow MLLMs with fine-grained visual understanding? Using our training recipe, we outperform SOTA open-source MLLMs on fine-grained visual discrimination with ClipCap, a mere 200M param simplification of modern MLLMs!!! 🚨Introducing No Detail Left Behind:

Check out our recent work on identifying the limitations and properties of SAEs! We use formal languages as a synthetic testbed to evaluate the methodology and suggest further steps.

New paper alert! 🧵👇 We show representations of concepts seen by a model during pretraining can be morphed to reflect novel semantics! We do this by building a task based on the conceptual role semantics "theory of meaning"--an idea I'd been wanting to pursue for SO long! 1/n