Interesting how in the Mistral-Large release they don't use the CoT@32/CoT@8 results for Gemini Pro (and no Gemini Ultra results), but presumably recompute them with no multiple CoT? I wonder if they tried to multiple CoT techniques with Mistral.

Explanation for the copilot issues recently: I think the decoding being used for copilot forces emojis at the end of a message Degeneration occurs since the decoding distribution is different from the 'correct' distribution. Implications: decoding strategies are exploitable

Lots of interest in diffusion on discrete spaces. I think Aaron's approach is the most principled I've seen. Plus, lots of applications in conditional generation!

Has anyone come up with good ideas on how to use LLM assistants, given adversarial inputs? I've already read simonwillison.net/2023/Apr/25/du…, any more discussion?

Reading the report about many-shot jailbreaking reminds me of the paper from Wolf et al last year: under weak assumptions then for sufficiently long context, any LLM is jailbreakable. arxiv.org/pdf/2304.11082…

As part of our ongoing work on AI safety and security, we've discovered a powerful, yet simple LLM jailbreak that exploits an intrinsic LLM behavior we call 'crescendo' and have demonstrated it on dozens of tasks across major LLM models and services: …ndo-the-multiturn-jailbreak.github.io

Happy to announce I was one of the winners of the OpenAI preparedness challenge! If anyone has a safety-related project that is bottlenecked on OpenAI credits, let me know--I'd be happy to help out openai.com/blog/frontier-…

We need to be spending more money on evals to get better measurements of accuracy and confidence. I wrote about how to compare model performance with the most basic statistical tools. Although these tools don't capture all the nuance, it's better than the status quo.

🚀 Excited to share our latest #AI research from Stanford on privacy-constrained reinforcement learning, developed with Chris Cundy and Stefano Ermon! Our framework minimizes sensitive information exposure using mutual information regularizers. 🤖💡#AISTATS2024 1/6

I'm at ICLR this week -- If you're there too, let's chat! Interested in hearing more about sequence models and LLMs, particularly about alignment, detecting deception, and scalable oversight

Yesterday was my last day as head of alignment, superalignment lead, and executive OpenAI.

When you leave OpenAI, you get an unpleasant surprise: a departure deal where if you don't sign a lifelong nondisparagement commitment, you lose all of your vested equity: vox.com/future-perfect…

Improving to 60% on GPQA from the previous SoTA of 54% is *really* impressive -- the GPQA questions are very difficult! (That is, assuming no test-set leakage...)

Life update: I'm excited to announce that I defended my PhD last month and have joined FAR.AI as a research scientist! At FAR, I'm investigating scalable approaches to reduce catastrophic risks from AI.