It's with enormous pleasure that I tell you that our last paper has been accepted for publication in the IEEE TPAMI, the most important Machine Learning journal. Check it out! ieeexplore.ieee.org/document/96614… Battista Biggio Stefano Melacci Angelo Sotgiu Marco Gori Ambra

📣 Last week I presented "Foveated Neural Computation" ECML PKDD ➡️ Paper: 2022.ecmlpkdd.org/wp-content/upl… ➡️ Code: github.com/sailab-code/fo… A thread👇🧵 In some visual tasks, such as image classification, salient information is distributed in regions of limited size 🧐 1/n

See you all in Pisa 😃 !!!!!! (July 2024)

🚨1 month left until the #CoLLAs2024 paper submission deadline!🚨 Don’t miss your opportunity to contribute to the field of lifelong learning. Finalize your research and submit by 15 Feb 2024! 📄 Prepare your submission: lifelong-ml.cc/Conferences/20… 🗓️ Abstract deadline: 09 Feb 2024

📢📢📢The abstract deadline for #CoLLAs2024 is tomorrow 👇

‼️Short survey paper‼️ 📢"On the Resurgence of Recurrent Models for Long Sequences: Survey and Research Opportunities in the Transformer Era" by Matteo Tiezzi M.Casoni Alessandro Betti tommaso guidi Marco Gori Stefano Melacci #Transformers #SSMs arxiv.org/abs/2402.08132

🧐Interested in the details of the latest SOTA architectures (#RWKV , #RetNet, #Mamba, #Griffin) for long sequences? 📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" arxiv.org/abs/2406.09062

Goal: efficiently 𝐩𝐫𝐨𝐜𝐞𝐬𝐬 𝐥𝐨𝐧𝐠 𝐬𝐞𝐪𝐮𝐞𝐧𝐜𝐞𝐬 + 𝐩𝐫𝐞𝐬𝐞𝐫𝐯𝐞 𝐥𝐨𝐧𝐠 𝐭𝐞𝐫𝐦 𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐜𝐢𝐞𝐬 TL;DR #Transformers + #RNNs, #DeepStatespace models ➡️ progressive confluence of previously disjoint architectures arxiv.org/abs/2406.09062

We summarized the key ingredients underlying the current models generation: *️⃣ linear recurrence *️⃣ element-wise recurrence *️⃣ gating mechanisms and where there might be more room for novel research activities: #onlinelearning arxiv.org/abs/2406.09062

We described several trends: 1️⃣ #Transformers embraced Recurrence exploiting a stateful recurrent representation: *️⃣ all the latest models RWKV4/5/6 by BlinkDL *️⃣ #RetNet by Li Dong et al. are competitive with quadratic self-attention! arxiv.org/abs/2406.09062

3⃣ RNNs evolutions, such as xLSTMs (Günter Klambauer), ODE-inspired RNNs (Neural Oscillators - Konstantin Rusch) as well as novel paths for 4⃣ Learning in RNNs, such as modern evolutions of RTRL (Khurram Javed) or bio-inspired methods (Beren Millidge) arxiv.org/abs/2406.09062

We also summarized the most common #benchmarks and settings for long sequence processing and #longrange dependencies arxiv.org/abs/2406.09062

📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" arxiv.org/abs/2406.09062 a joint work by Matteo Tiezzi , M. Casoni, Alessandro Betti , M.Gori (Marco Gori) and Stefano Melacci

Hurry up for CoLLAs 2025 registration!

#CoLLAs2024 I will present our novel architectural solution for Continual Learning, Memory Head, at the upcoming CoLLAs 2025 in Pisa! Paper and video: lifelong-ml.cc/Conferences/20… 📝Pass by at our poster on Tuesday 30th and to the oral presentation on Wednesday 31th morning!

Heat and social dinner CoLLAs 2025 #collas2024

🚀 We’ve launched a new site for Collectionless AI, sharing our take on next-gen AI/ML! 🚀 Explore our papers, events, and more! cai.diism.unisi.it #CollectionlessAI #ContinualLearning #LifelongLearning #MachineLearning #AI 🌐 Stefano Melacci Marco Gori Matteo Tiezzi

The call for two post-doc positions in #CollectionlessAI is open (University of Siena, Italy)! Have a look at the link in this post 😀 #CollectionlessAI #ContinualLearning #LifelongLearning #MachineLearning #AI Stefano Melacci Marco Gori cai.diism.unisi.it/post/post_doc_…

🚀 Learning Over Time (LOT) Spring School 📅 24-27 March 2025 | 📍 Siena, Italy 🔗sites.google.com/unisi.it/lot-s… Organizers: Stefano Melacci ,Vincenzo Lomonaco Andrea Cossu ,Alessandro Betti Matteo Tiezzi #ContinualLearning #CL #LifelongLearning #CollectionlessAI

The first edition of Learning Over Time (LOT) Spring School is starting! Stefano Melacci kicking off the school at the wonderful Certosa di Pontignano, Italy Organized by Stefano Melacci, Vincenzo Lomonaco , Andrea Cossu , Alessandro Betti , and myself cai.diism.unisi.it/school/