We don't expect Bayesian methods to do so well at large scale, but we can now get decent improvements with variational learning to GPT-2. I wrote a blog about this (first one in a long time). Check it out! team-approx-bayes.github.io/blog/ivon/ Paper: arxiv.org/abs/2402.17641 A thread below.

東京楽しいよ！

What a day we got to spend in the presence of a legend. Prof. Amari is 88 now and we heard, what he calls, his last talk today (but he is in very good health and I am sure there will more).

Just wrote a (math-free) tutorial for the IVON optimizer proposed in our paper "Variational Learning is Effective for Large Deep Networks". Maybe it is just what you need for your next DL project 😉 ysngshn.github.io/research/why-i…

arxiv.org/abs/2402.17641 Our paper has been accepted to ICML🎉 more coming soon!

Model Merging has shown great success but key questions remain unresolved: ✅ Why does it work? ❌ When can it fail? We shed light on those questions by connecting inaccuracies of weighted-averaging to mismatches in the gradients. 🧵(1/9) #ICLR2024 📰 arxiv.org/abs/2310.12808

Come to our poster, Halle B #297 in afternoon session today. #ICLR2024

RIKEN Center for Advanced Intelligence Project ABI team reunion at ICLR 2024, with Nico Daheim Alexandre L.-Piché Thomas Möllenhoff Erik Happy B. Kasumba Anshuk Uppal and Naima

Amazing location and great to have a major conference in Asia again! 🎉👏

It is my pleasure to announce the 2nd Bayes-Duality workshop, focusing on the design of AI that learns adaptively, robustly, and coninually, like humans. bayesduality.github.io/workshop_2024.… You can watch all 26 talks through zoom livestream (June 12-21). Register at …c59ed978213830355fc8978.doorkeeper.jp/events/172217

Looking forward to the talk! Register here to join in remotely: …c59ed978213830355fc8978.doorkeeper.jp/events/173934

I‘m in Venice at ISBA this week. I‘ll be giving a talk on making variational Bayes work for large neural nets (GPT-2, …), Friday 3pm in Julyan Arbel invited session. It‘s my first time at ISBA, happy to chat and connect. Send me a message! 😊

Great ISBA session on Bayesian computations. Healthy mix of speakers Florence Forbes Thomas Möllenhoff Hien Nguyen newcomers to ISBA & nice handwritten discussion by Judith Rousseau who was a newcomer in the previous century :)

I‘ll be presenting a poster on our work on scaling variational learning to GPT-2 sized models today at ICML. Feel free to drop by!

The JAX implementation of IVON is now available at github.com/ysngshn/ivon-o… Let us know if you have any issues or success stories to share! #JAX #IVON #deeplearning

#CoLLas2024 Stefano Melacci introducing the fourth keynote speaker, Emtiyaz Khan with the talk "How to make machines that adapt quickly"

Thanks JinYeong Bak and students for visiting us at RIKEN and giving excellent talks! If you haven’t seen it yet, check out their recent works on memory in neural nets. arxiv.org/abs/2310.03052 aclanthology.org/2024.naacl-lon…

Next Thursday, Sept. 12: Seminar on Advances in Probabilistic Machine Learning returns, with Thomas Möllenhoff: Variational Learning is Effective for Large Deep Networks. Register here for the Zoom link: aaltoml.github.io/apml/ cc Arno Solin Computer Science Computer Science @ University of Helsinki RIKEN Center for Advanced Intelligence Project

Thanks Kai Arulkumaran for the inspiring talk on brain-controlled robotics and visiting us at RIKEN AIP. Looking forward to what is coming next 🤖⚡️

We have two open post-doc positions. You dont' have to be a Bayesian but somebody who is interested to work with at the intersection of DL, Bayes, and optimization. riken.jp/en/careers/res… Interest in understanding deep learning and continual lifelong learning is a plus!