Congratulations to Australia on winning the World Cup! Great run to the finals, India.

I'll be at #NeurIPS2023 next week! Very excited to present Tree-ring (poster session 1, #126) and PEZ (poster session 4, #606)! Feel free to DM me if you would like to chat about ML privacy and security or even just want to grab some hotpot🍲 together!

I will be at Neurips'23, presenting our work on Cold Diffusion. Excited for a week packed with learning, awesome discussions, and yes, jazz tunes to end the days. #NeurIPS2023

🔍 Do diffusion models have to reply on reversing ‘Gaussian degradation’? Not necessarily. Check out ‘Cold Diffusion’, which reverses deterministic image degradations. It paves the way for generalized diffusion models that invert arbitrary processes. #ML #AI #DiffusionModels

We train the networks in two stages. Our design follows the generalized diffusion model framework from Cold Diffusion Arpit Bansal Tom Goldstein, so we can directly use their sampling algorithm. Essentially, this leads to an alternation of forecasting and interpolation. 3/7

Will be presenting at the evening poster session from 5-7 pm in Hall B1 + B2, at booth #1918. Feel free to come by for a discussion! #NeurIPS23

Our work on "Universal Guidance for Diffusion Models" has been accepted to ICLR2024. TLDR - The tweet features representations of my identity in various styles. #ICLR2024

the math in most diffusion papers

Just got back from vacation, and super excited to finally release Griffin - a new hybrid LLM mixing RNN layers with Local Attention - scaled up to 14B params! arxiv.org/abs/2402.19427 My co-authors have already posted about our amazing results, so here's a 🧵on how we got there!

We show how to make data poisoning and backdoor attacks way more potent by synthesizing them from scratch with guided diffusion. 🧵 1/8 Paper: arxiv.org/abs/2403.16365

Transformers Can Do Arithmetic with the Right Embeddings The poor performance of transformers on arithmetic tasks seems to stem in large part from their inability to keep track of the exact position of each digit inside of a large span of digits. We mend this problem by

In this paper, we investigate arithmetic to enhance and expand the generalization capabilities of transformers. Our study demonstrates excellent OOD performance, marking a significant effort towards building neural networks that can outperform their training data.

We often determine whether a neural network is over or under parameterized by counting parameter. In practice, how much data we can fit depends on many factors: architecture, optimizer, etc. So just how flexible are neural networks in practice? 🧵 Paper: arxiv.org/abs/2406.11463

We're presenting Abacus Embeddings 🧮 today from 10:10-11:10 in Lehar 3 at the ICML Workshop on Large Language Models and Cognition. Drop by to see how we can improve your language model.

📢I’ll be admitting multiple PhD students this winter to Columbia University 🏙️ in the most exciting city in the world! If you are interested in dissecting modern deep learning systems to probe how they work, advancing AI safety, or automating data science, apply to my group.

I wish everyone a very Happy Diwali 2024! I hope it brings peace to everyone and fills it with lights and colors. A throwback to my 2019 Diwali in India.

Favorite part of PhD Thesis - Writing and Reading other's Acknowledgments.

Lol, Deepseek won't list all the states of India.

Graduation complete!