The next chapter about transformers is up on YouTube, digging into the attention mechanism: youtu.be/eMlx5fFNoYc

The model works with vectors representing tokens (think words), and this is the mechanism that allows those vectors to take in meaning from context.

😯The 100% jailbreaking of GPT3.5/4, Llama-2, Gemma, and Claude2/3 is truly impressive! Smart ideas for leveraging API access.

Congrats to Maksym Andriushchenko 🇺🇦 francesco croce TML Lab (EPFL) !

arxiv.org/abs/2404.02151

Thrilled to announce the Trustworthy Interactive Decision-Making with Foundation Models workshop at #IJCAI2024 ! 🌟 A great opportunity for dialogue on ethical AI decision-making, blending human-centric methods & foundation models.

‼️⚠️bad day to be a LLM⚠️‼️

Haize Labs took one of our favorite adversarial attack algorithms, GCG, and made it *38x* faster

Two of the best elaborate parody sites in recent memory are on exactly the opposite sides of the AI safety debate.

Open Asteroid Impact (openasteroidimpact.org) and Goody 2 (goody2.ai) which seems to have an actual LLM.

We need more debates via maximalist parody.

Mix and Match Your LLMS To Avoid A High Latency Hellscape

Most LLM apps and AI agents need multiple calls to an LLM. As many as 5-10 calls are required for a moderately complex LLM app or AI agent. Calling GPT-4 or Claude is highly impractical, and you will soon be in

Introducing Maestro ✨

A framework for Claude Opus to orchestrate subagents.

Simply ask for a goal, and Opus will break it down and intelligently orchestrate instances of Haiku to execute subtasks, which Opus will review at the end. 🧙‍♂️

The output is saved as markdown 👇

Some text data is private & cannot be shared... Can we generate synthetic replicas with privacy guarantees?🤔

Instead of DP-SGD finetuning, use Aug-PE with inference APIs! Compatible with strong LLMs (GPT-3.5, Mistral), where DP-SGD is infeasible.
🔗alphapav.github.io/augpe-dpapitext [1/n]

Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️

It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵 dpmd.ai/3TiYV7d

Can hazardous knowledge be unlearned from LLMs without harming other capabilities?

We’re releasing the Weapons of Mass Destruction Proxy (WMDP), a dataset about weaponization, and we create a way to unlearn this knowledge.

📝arxiv.org/abs/2403.03218
🔗wmdp.ai

Independent AI research should be valued and protected.

In an open letter signed by over a 100 researchers, journalists, and advocates, we explain how AI companies should support it going forward.

sites.mit.edu/ai-safe-harbor/

1/

💯💯💯🎇

Large Vision Language Models are prone to object hallucinations – how to cost-efficiently address this issue? 🚀 Introducing MARINE: a training-free, API-free framework to tackle object hallucinations.

Joint work with an amazing team Linxi Zhao Weitong ZHANG and Quanquan Gu!

Wondering why LLM safety mechanisms are fragile? 🤔
😯 We found safety-critical regions in aligned LLMs are sparse: ~3% of neurons/ranks
⚠️Sparsity makes safety easy to undo. Even freezing these regions during fine-tuning still leads to jailbreaks
🔗 boyiwei.com/alignment-attr…
[1/n]

just found out this library. incredibly easy to get Mixtral ready
github.com/hiyouga/LLaMA-…

Increasingly finding that the one thing that most makes managers panic about AI is showing them, not the advanced features of GPT-4, but rather the fact that Copilot for Office can create an OK PowerPoint with speaker notes from a document in 47 seconds.

This is all real time.

🎉 Two of my papers have been accepted this week at #ICLR2024 & #AISTATS !
Big thanks and congrats to co-authors Xuxi Chen & Eric Gan, mentors Atlas Wang & Gintare Karolina Dziugaite, and especially my advisor Baharan Mirzasoleiman! 🙏
More details on both papers after the ICML deadline!

LLMs can be backdoored easily, even with the use of chain of thought, which we show can be turned into another weakness.
Our paper BadChain accepted by ICLR provides the first backdoor attack against LLMs with COT prompting. openreview.net/pdf?id=S4cYxIN…

Our paper on ✨ universal LLM backdoors ✨ through RLHF poisoning has been accepted at ICLR 2024 2024. See you in Vienna! #ICLR2024

🌶️ Uncovering the potential risks of LLM customization via finetuning, accepted as an oral (1.2%) at ICLR 2024’24

Couldn't be prouder of my team!
🌐: llm-tuning-safety.github.io.

(we also studied LLM backdoors and its’ implications, Evan Hubinger Anthropic, consider citing us 😕)