How to identify bias in language agency?Eg. in texts describing White men as “leading” & Black women as “helping”?🧐
🔎String matching?❌NO!
🔎Sentiment classifier?❌No!
✅Our agency classifier CAN! It reveals gender, racial, and intersectional bias🤯
🔗:
arxiv.org/abs/2404.10508

As we increasingly rely on #LLMs for product recommendations and searches, can companies game these models to enhance the visibility of their products?

Our latest work provides answers to this question & demonstrates that LLMs can be manipulated to boost product visibility!…

🚀 Exciting news! Our latest work, CHM-Corr++, has been accepted for presentation at the #XAI4CV Workshop, CVPR 2024! 🎉

The work lies in the intersection of: Interactive XAI and human-AI collaboration.

Demo: http://137.184.82.109:7080/
Paper: arxiv.org/abs/2404.05238

🚨 Are leading safety-aligned LLMs adversarially robust? 🚨

❗In our new work, we jailbreak basically all of them with ≈100% success rate (according to GPT-4 as a semantic judge):
- Claude 1.2 / 2.0 / 2.1 / 3 Haiku / 3 Sonnet / 3 Opus,
- GPT-3.5 / GPT-4,
- R2D2-7B from…

Thanks LLM Security for sharing our new #ICLR2024 work 'Can LLM-Generated Misinformation Be Detected?'

🔗Project website (paper, dataset, and code): llm-misinformation.github.io

🚨LLM-generated misinformation is one of the most critical risks on AI safety. Then, one fundamental…

Are you interested in jailbreaking LLMs? Have you ever wished that jailbreaking research was more standardized, reproducible, or transparent?

Check out JailbreakBench, an open benchmark and leaderboard for Jailbreak attacks and defenses on LLMs!

jailbreakbench.github.io
🧵1/n

We're recruiting two Research Assistants to join us and work on the security of ML-based personal assistants at
Imperial College London. The role will focus on verification, robustification and adversarial attacks for AI assistants. rb.gy/mcxvob.

Can we adaptively generate training environments with LLMs to help small embodied RL game agents learn useful skills that they are weak at? 🤔

👉 Check out EnvGen, an effective+efficient framework in which an LLM progressively generates and adapts training environments based on…

The U.S. Supreme Court has ended the use of race in college admissions. Fortunately, there exists a path to fair algorithmic decision-making that differs from the invalidated affirmative action measures, as we discuss in our recent Uncommon Good post:
uncommongood.substack.com/p/fair-machine…

We are excited to present a new event of our seminar series on ML Security!
We will host Giovanni Cherubin (@Microsoft) on March 26th, 2024 at 15:00 CET.
Free registration: us02web.zoom.us/j/82941308293?…

ELSA - European Lighthouse on Secure and Safe AI Adversarial Machine Learning Trustworthy ML Initiative (TrustML) AI Village @ DEF CON RedTeamVillage

Wanna know gpt-3.5-turbo's embed size? We find a way to extract info from LLM APIs and estimate gpt-3.5-turbo’s embed size to be 4096. With the same trick we also develop 25x faster logprob extraction, audits for LLM APIs, and more!
📄 arxiv.org/abs/2403.09539
Here’s how 1/🧵

🤔Can LLM agents really simulate human behaviors?

🌟Our new paper 'Can Large Language Model Agents Simulate Human Trust Behaviors?' (Project website: camel-ai.org/research/agent…) provides some new insights into this fundamental problem.

✨TLDR: We discover the trust behaviors of…

🧐Are LLM responses to public health questions biased toward specific demographic groups?

In our new interdisciplinary collaboration, we find that disparities exist among model answers for different groups across ages, U.S. locations, and sexes.

Paper: arxiv.org/pdf/2403.04858…

The final layer of an LLM up-projects from hidden dim —> vocab size. The logprobs are thus low rank, and with some clever API queries, you can recover an LLM’s hidden dimension (or even the exact layer’s weights).

Our new paper is out, a collaboration between lot of friends!

Just one month left before SaTML Conference April 9-11 in Toronto! I am excited to hear from Somesh Jha Deb Raji Yves-A. de Montjoye Sheila McIlraith, as well as the authors of accepted papers, and the competition organizing teams!

There's still time to register! satml.org

Our first Uncommon Good post (with Nick Perello) discusses how to train AI systems that do not propagate discrimination, in compliance with legal provision, based on our research published in ACM FAccT, AI, Ethics, and Society Conference (AIES), and ICML Conference. Stay tuned!
open.substack.com/pub/uncommongo…

Pointing to an image region should help models focus, but standard VLMs fail to understand visual markers/prompts (e.g., boxes/masks).

🚨Contrastive Region Guidance: Training-free method that increases focus on visual prompts by reducing model priors.

arxiv.org/abs/2403.02325
🧵

We are announcing the winners of our Trojan Detection Competition on Aligned LLMs!!

🥇 TML Lab (EPFL) (@fra__31, Maksym Andriushchenko 🇺🇦 and Nicolas Flammarion)
🥈 Krystof Mitka
🥉 nev

🧵 With some of the main findings!

LLMs are great, but their internals are less explored. I'm excited to share very interesting findings in paper

“Massive Activations in Large Language Models”

LLMs have very few internal activations with drastically outsized magnitudes, e.g., 100,000x larger than others. (1/n)

Thrilled to be recognized with best paper honorable mention at AAAI!

Our paper raises serious questions re: reproducibility + reliability in fairness

We define + mitigate arbitrariness, & find that most fairness benchmarks are actually close-to-fair

This is a BIG 🚩🚩

1/