Hi🍁 Vancouver and AAAI ! I'll be presenting our work on protecting fairness when learning from synthetic data, at the Poster Session tomorrow 7PM! Excited to chat with yall :) Thanks USC Thomas Lord Department of Computer Science for the great article on our work: viterbischool.usc.edu/news/2024/02/d…! #AAAI2024 #AAAI24

DM me if you're interested in: 🐋creating high-quality pretraining datasets 🐊studying data's impact on LM capabilities 🦉tools for sensemaking over large corpora 🐡adapting LMs to specialized domains like science 🐈evaluation through human interaction

If anything, what we're seeing is that internal red-teaming and alignment is NOT the miracle safety solution to everything in AI (very limited, biased & easy to jailbreaks) Truth is the safest way to build AI is openly, transparently and iteratively with the community.

When vision-language models are uncertain about their answers, abstaining (“I don’t know”) enhances system reliability, but at the cost of utility. We introduce ReCoVERR (arxiv.org/abs/2402.15610) to mitigate over-abstention in VLM systems without sacrificing prediction accuracy.

Introducing 🌈 Rainbow Teaming, a new method for generating diverse adversarial prompts for LLMs via LLMs It's a versatile tool 🛠️ for diagnosing model vulnerabilities across domains and creating data to enhance robustness & safety 🦺 Co-lead w/ Sharath Raparthy & Andrei Lupu

Using our Open Instruct and Tulu 2, we adapt OLMo to acquire different capabilities and safety measures through fine-tuning and Direct Preference Optimization (DPO). The adapted models demonstrate quick improvement on popular reasoning tasks such as MMLU and TruthfulQA, and on

I'm starting a blog! First post is on the AAAI conference, with some notes from the LLM panel, 10 cool papers, and some areas for improvement! cylumn.com/notes/aaai2024 #AAAI2024 #AAAI24

Sometimes, drinking coffee makes me feel like there are no bad ideas

bummer. I'm skeptical that this meaningfully reduces class sizes or saves the school money. just hurts students who want rigorous course loads.

Let’s talk about social simulations! Do you know that term could refer to various settings? Our new work suggests that you might want to double-check before being “amazed” by those simulations. 📜: arxiv.org/abs/2403.05020 🌐: agscr.sotopia.world 1/

Learn more about LLMS understanding language on homelessness and suicides on Swabha Swayamdipta's presentation at ShowCAIS on April 19th! More info: sites.google.com/usc.edu/showca… USC Viterbi School USC Social Work

Can GPT-4V and Gemini-Pro perceive the world the way humans do? 🤔 Can they solve the vision tasks that humans can in the blink of an eye? 😉 tldr; NO, they are far worse than us 💁🏻‍♀️ Introducing BLINK👁 zeyofu.github.io/blink/, a novel benchmark that studies visual perception

Curious about socially-intelligent AI? Check out our paper on underlying technical challenges, open questions, and opportunities to advance social intelligence in AI agents: Work w/ LP Morency, Paul Liang 📰Paper: arxiv.org/abs/2404.11023 💻Repo: github.com/l-mathur/socia… 🧵1/9

AI agents are increasingly social!! 🚀🚀🚀 Those “social agents"—including chatbots, web agents, and robots are becoming part of our everyday lives. We did an initial survey of recent social agent works 🤖✨, to synergize multiple domains for more socially aware AI. 1/

If you're interested in exploring or building upon the Sotopia environment to study social intelligence in language agents, check out the new Colab tutorial below from Hao Zhu 朱昊! 💻 More information on Sotopia is here: sotopia.world

We are pleased to announce that Maja Matarić of @usc , will be the 2024-2025 ACM Athena Lecturer! Matarić is recognized for pioneering the field of socially assistive #robotics. Learn more abt her impact here: bit.ly/4bF71P4 #womenincomputing @csatusc @uscviterbi

🩺Introduce Knowledge Bottlenecks: incorporating priors 💡 from medical documents 📚 through inherently interpretable models. KnoBo is robust to domain shifts in medical images, such as data sampled from different hospitals 🏥 or data confounded by demographic variables such as

Our work on improving selective prediction for VLMs has been accepted to #ACL2024 Findings! Read on to learn how you can make your VLM both reliable *and* usable ✨ Paper: arxiv.org/abs/2402.15610 Code: github.com/tejas1995/ReCo…

You think your model just fell out of a coconot tree 🥥? It should not always comply in the context of all it has seen in the request. Check out our paper on contextual noncompliance.

GPT4-o1-preview from OpenAI now gets 80.4% (compared to 14% performance of GPT4o) on the Connections game in 1 single attempt. Saw a thread on LinkedIn about similar bump on Wordle. I also attached some other models in comparison. This is very impressive given how hard the task