I am seeking a paid research collaborators for two months to study AI-generated misinformation in India.

If you have worked on creating such misinformation, or have experience studying deepfakes and/or folks who generate such content, please contact me.

Should we trust LLM evaluations on publicly available benchmarks?🤔

Our latest work studies the overfitting of few-shot learning with GPT-4.

with Harsha Nori Vanessa Rodrigues Besmira Nushi 💙💛 and Rich Caruana

Paper: arxiv.org/abs/2404.06209

More details👇 [1/N]

Today Meta released Llama 3! Congrats to the team.

In their blog post they wrote that, 'the curation of a large, high-quality training dataset is paramount', while providing almost no information about how it was made, how it was filtered, or its contents.

📢 New Paper Alert! 📝 'Explaining Explainability: Understanding Concept Activation Vectors' 📄arxiv.org/abs/2404.03713

What does it mean to represent a concept as a vector?

We explore three key properties of concept vectors and how they can affect model interpretations. 🧵(1/5)

<Summer Research Internship at KIXLAB>
I'm looking for undergrad research interns to join my research group (kixlab.org) this summer. Most projects this round are about human-AI interaction. Please share broadly!

juhokim.com/2024-summer-in…

github.com/microsoft/mech… very excited about releasing our new repo on mechanistically understanding factual errors of LLMs. This is also the codebase for the new #ICLR2024 paper 'Attention satisfies: A constraint-satisfaction lens on factual errors of language models' by

Full lecture slides and reading list for Roger Grosse's class on AI Alignment are up:
alignment-w2024.notion.site

arxiv.org/abs/2402.14735

Lujo (Lujo Bauer) and I are seeking nominations for service on the program committee for USENIX Security '25. You may nominate yourself or someone else by Friday, May 24, 2024: forms.gle/VAWFhYzuBQo6kP….

Want to give a talk at HotPETS this year? Here's how to apply! petsymposium.org/2024/hotpets.p…

The deadline is May 14, 2024.

Hot Topics in Privacy Enhancing Technologies (HotPETs) is the discussion-oriented workshop at PETS.

Please retweet! #pets24 #hotpets24 #privacy

1/n

Excited to share this work (to be presented in STOC 2024), which provides new and improved ways to delegate machine-learning tasks via PAC verification, a beautiful notion recently introduced by Goldwasser, Rothblum, Shafer, and Yehudayoff. 1/2

arxiv.org/abs/2404.08158…

The new SaTML pc chairs Konrad Rieck 🌈 and Somesh Jha are looking for a general chair & venue for the 2025 conference.

If you're interested in hosting the conference in April 2025 (the exact date/month is flexible), submit a bid here:

tinyurl.com/hostsatml

Soft deadline: May 15 2024

Grokking as the Transition from Lazy to Rich Training Dynamics ift.tt/tVmRI6Q

Being able to interpret an #ML model’s hidden representations is key to understanding its behavior. Today we introduce Patchscopes, an approach that trains #LLMs to provide natural language explanations of their own hidden representations. Learn more → goo.gle/4aS5epd

In-context learning (ICL) circuits emerge in a phase change...

Excited for our new work 'What needs to go right for an induction head (IH)?' We present 'clamping', a method to causally intervene on dynamics, and use it to shed light on IH diversity + formation.

Read on 🔎⏬

Introducing Rerank 3! Our latest model focused on powering much more complex and accurate search.

It's the fastest, cheapest, and highest performance reranker that exists. We're really excited to see how this model influences RAG applications and search stacks.

Exploring Concept Depth: How Large Language Models Acquire Knowledge at Different Layers?

repo: github.com/Luckfort/CD
abs: arxiv.org/abs/2404.07066

The first SaTML Conference tutorial by Yves-A. de Montjoye on detecting copyright material in LLMs is now starting!

Aligning LLMs to Quote from Pre-Training Data

Interesting approach to enable verifiability in LLMs.

Proposes techniques to align LLMs to leverage memorized information quotes directly from pre-training data.

Looks like the alignment approach is not only able to generate

'Now, Later, and Lasting: 10 Priorities for AI Research, Policy, and Practice,' forthcoming in CACM, with Vincent Conitzer Sheila McIlraith Peter Stone
#AI100 Communications of the ACM Association for Computing Machinery Stanford HAI Partnership on AI White House Office of Science & Technology Policy Carnegie Mellon University University of Toronto CIFAR arxiv.org/abs/2404.04750