Excited to announce that I'll be joining NEU CS Northeastern U. Khoury College of Computer Sciences as an Assistant Professor in Fall 2025! 🎉 I'm looking for PhD students to work with me on the theoretical foundations of trustworthy ML. If interested, apply by Dec 15! More info at

Did anyone else not receive any LLM feedback on their ICLR reviews? I was looking forward to this.

Presenting this work today with Dutch Hansen at 11am in West Ballroom A-D #5803! Swing by if you are around!

Come see Julian talk about compact metric spaces, finite projections, and sample complexity of learning algorithms!

the best indicator for whether you should do a PhD is that you’re hell bent on doing it regardless of what others say. if you are the kind of person who constantly looks for reasons not to do the PhD (eg people on twitter saying LLMs obviate the need for a PhD), don’t do a PhD

Great to see others discovering similar findings as we did in our Neurips2024 paper (arxiv.org/abs/2406.03445). We call these Fourier features instead of helix. How are these features useful for representing numbers? Stay tuned for our new number embedding paper coming soon!

Our work (with Tianyi Zhou Deqing Fu and Vatsal Sharan) published at NeurIPS 2024 already found that pretrained LMs do addition using modular arithmetic/“trigonometry” (we called these Fourier features). Indeed it is a clever mechanism.

On Agnostic PAC Learning in the Small Error Regime ift.tt/xgeHsF0

Billion-parameter LLMs still struggle with simple arithmetic? 📞 FoNE (Fourier Number Embedding) tackles this problem. By mapping numbers directly into Fourier space, it bypasses tokenization and significantly improves numerical accuracy with better efficiency and accuracy.

😋 Want strong LLM reasoning without breaking the bank? We explored just how cost-effectively RL can enhance reasoning using LoRA! [1/9] Introducing Tina: A family of tiny reasoning models with strong performance at low cost, providing an accessible testbed for RL reasoning. 🧵

Textual steering vectors can improve visual understanding in multimodal LLMs! You can extract steering vectors via any interpretability toolkit you like -- SAEs, MeanShift, Probes -- and apply them to image or text tokens (or both) of Multimodal LLMs. And They Steer!

Sparse autoencoders (SAEs) can be used to elicit strong reasoning abilities with remarkable efficiency. Using only 1 hour of training at $2 cost without any reasoning traces, we find a way to train 1.5B models via SAEs to score 43.33% Pass@1 on AIME24 and 90% Pass@1 on AMC23.