Dear Students, please don't get discouraged... we need you. Quoting another Geoff: "The future depends on some graduate student who is deeply suspicious of everything I have said" (8/8)

Proud to announce our AdaSubS algorithm, which is "notable-top-5%" on #ICLR2023! Kudos to Michał Zawalski Michal Tyrolski Konrad Czechowski who led the project and to all the other wonderful co-authors Tomek Odrzygóźdź @Dstauchur Piotr Piękos Yuhuai (Tony) Wu Łukasz Kuciński youtu.be/7GZbPB1Gu0E

In complex problems, states present varying degrees of difficulty, from easy, with obvious action sequences, to hard, requiring reasoning. Can planning methods adapt to this accordingly? That’s exactly what AdaSubS is doing! youtu.be/7GZbPB1Gu0E 🧵👇

1/x Proud to announce our AdaSubS: RL algorithm, which is "notable-top-5%" on `#ICLR2023`! Congratulations to the team of wonderful co-authors: Michał Zawalski, Konrad Czechowski Tomek Odrzygóźdź @Dstauchur Piotr Piękos Yuhuai (Tony) Wu Łukasz Kuciński Piotr Miłoś

The project I was involved in has been selected as "notable-top-5%" on #iclr2023, check it out! Congratulations to all authors and thanks for the fantastic collaboration: Michał Zawalski Konrad Czechowski Michal Tyrolski @Dstauchur Tomek Odrzygóźdź Yuhuai (Tony) Wu Łukasz Kuciński Piotr Miłoś

Our gloriously picked cherries. Their wicked failure cases.

Come visit our poster "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention" on Thursday at 11 am in East Exhibit Hall A-C on #NeurIPS2024. With Piotr Piękos, Kazuki Irie and Jürgen Schmidhuber.

One of my rejected ICML 2025 (ICML Conference) papers. Can anyone spot any criticism in the metareview? What a joke 🙃

What if instead of a couple of dense attention heads, we use lots of sparse heads, each learning to select its own set of tokens to process? Introducing Mixture of Sparse Attention (MoSA)

[LG] Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing P Piękos, R Csordás, J Schmidhuber [KAUST & Stanford University] (2025) arxiv.org/abs/2505.00315

Intuitive and reduces complexity. Measurable effectiveness. That's what you're going for.

Next time you complain about the bidding system, be grateful that you at least had a chance to bid.