Positional Attention: Expressivity and Learnability of Algorithmic Computation (v2) We study the effect of using only fixed positional encodings (referred to as positional attention) in the Transformer architecture for computational tasks. These positional encodings remain the

Are you a Canadian citizen or PR? 3rd or 4th year in undergrad? Available June 2-13 for a fully funded summer school in statistics? Learn more and apply here: statistics.utoronto.ca/UTSSRP Yours truly will be teaching you some online learning theory.

Our MIT class “6.S184: Introduction to Flow Matching and Diffusion Models” is now available on YouTube! We teach state-of-the-art generative AI algorithms for images, videos, proteins, etc. together with the mathematical tools to understand them. diffusion.csail.mit.edu (1/4)

BREAKING: Amii Chief Scientific Advisor, Richard S. Sutton, has been awarded the A.M. Turing Award, the highest honour in computer science, alongside Andrew Barto! Read the official Association for Computing Machinery announcement: hubs.la/Q039nZXM0 #TuringAward #AI #ReinforcementLearning

Congratulations to the latest Turing award winners Barto and Sutton! At this point, all three Canadian national AI institutes have a pioneering Turing winner in leadership: Amii - Rich Sutton Vector - Geoff Hinton Mila - Yoshua Bengio Canada AI haters real quiet right now.

.Shenghao Yang's Ph.D Thesis "Perspectives of Graph Diffusion: Computation, Local Partitioning, Statistical Recovery, and Applications" is now available. Link: dspacemainprd01.lib.uwaterloo.ca/server/api/cor… Relevant papers: 1) p-Norm Flow Diffusion for Local Graph Clustering: proceedings.mlr.press/v119/fountoula…

We are presenting 3 orals and 1 spotlight at #ICLR2025 on two primary topics: On generalizing the data-driven flow matching algorithm to jump processes, arbitrary discrete corruption processes, and beyond. And on highly scalable algorithms for reward-driven learning settings.

Discrete Flow Matching extends the Flow Matching recipe to discrete data. But so far the focus of the community has been on the simple masking corruption process. We now enable general corruption processes. Imagination is the limit! Oral by Neta Shaul Sat 3:30pm.

📣I'll be at the poster session with our follow-up on Discrete Flow Matching. We derive a closed-form solution to the kinetic optimal problem for conditional velocity on discrete spaces. Into flow models? come chat! 💬 🗓Poster: Sat 10am (#191), 🎤Oral: Sat 3:30pm (6E) #ICLR2025

Against conventional wisdom, I will be giving a talk with particular focus on the "how" and the various intricacies of applying stochastic control for generative modeling. Mon 9:50am Hall 1 Apex #ICLR2025 Also check out the other talks at delta-workshop.github.io!

Hard to describe how incredible Montreal is between May and November. Total vibe.

Announcing the newest releases from Meta FAIR. We’re releasing new groundbreaking models, benchmarks, and datasets that will transform the way researchers approach molecular property prediction, language processing, and neuroscience. 1️⃣ Open Molecules 2025 (OMol25): A dataset

We've open sourced Adjoint Sampling! It's part of a bundled release showcasing FAIR's research and open source commitment to AI for science. github.com/facebookresear… x.com/AIatMeta/statu…

(1/n)🚨You can train a model solving DFT for any geometry almost without training data!🚨 Introducing Self-Refining Training for Amortized Density Functional Theory — a variational framework for learning a DFT solver that predicts the ground-state solutions for different

The supervision signal in AI4Science is so crisp that we can solve very complicated problems almost without any data or RL! In this project, we train a model to solve the Schrödinger equation for different molecular conformations using Density Functional Theory (DFT) In the

Why are we hiding updated scores during NeurIPS rebuttals. What is this trying to fix lol

The news is out! We're starting Blank Bio to build a computational toolkit assisted with RNA foundation models. If you want to see my flip between being eerily still and overly animated check out the video below! The core hypothesis is that RNA is the most customizable molecule

Me: "Where did you go to school?" Person: "Depends, undergrad or PhD?" ... are you only allowed to tell me one of them?