Following the weekend buzz about long range sequence modeling, let me tell you about our work on S5: a simple state space layer that achieves top-level performance and efficiency with parallel scans. (E.g. 98.58% on Path-X!) 🧵[1/12]

We're excited to keep building on S5! Personally, I see lots of applications in computational neuroscience, where we are often interested in predicting one sequence from another (e.g. behavioral outputs from neural spike trains). Stay tuned for more soon! [12/12]

Cool work and impressive results! While S4 and most variants are closer to CNNs, S5 can be seen as maximizing its potential as an RNN. This can unlock creative applications, like examining the hidden states or addressing irregular sampling. Many more cool ablations in the paper!

📢 Happening Today! 📢 Come learn about SIXO, our method for smoothing SMC which received an oral at #Neurips2022 (poster 1040, Thursday 4pm). SIXO provides a generic and powerful way to leverage future information when proposing and resampling particles in an SMC sweep. 🧵[1/N]

🚨🚨🚨#TWEEPRINT TIME🚨🚨🚨 @leaduncker and I are thrilled to share our investigation of motor cortical dynamics using optogenetic and electrical perturbations, now on bioRxiv Neuroscience biorxiv.org/content/10.110… Full thread at djoshea.notion.site/Twitter-Thread…

With co-authors @xulunasun, Saurabh Vyas, Eric Trautmann, Ilka Diester, Charu Ramakrishnan, with Krishna Shenoy and Maneesh Sahani, and essential support from Karl Deisseroth and the lab (Ofer Yizhar 💔, liefefenno!). Thanks to A Numrikko, I Ozden, J Wang for co-ax optrodes!

We used neural perturbations to identify the dynamics in the primate motor cortex that govern reaching movements. We deliver optogenetic excitation and electrical microstimulation (ICMS) to perturb neural states, record the surrounding neurons with electrodes and Neuropixels.

𝗞𝗲𝘆 𝘁𝗮𝗸𝗲𝗮𝘄𝗮𝘆𝘀: (𝟭) We reveal that task-related dynamical sensitivity is restricted to a self-contained, low-dimensional subspace of the ambient high-dimensional neural circuit, which has implications for multi-task learning, interference, noise robustness, etc.

(𝟮) We show that the task dynamics space ≠ the task activity space identified via PCA (or TDR). The task dynamics space is actually oriented so as to be robust to strong non-normal amplification within cortical circuits.

(𝟯) Lastly, we identify a fundamental difference in how broad, unstructured optogenetic excitation (in this context) and electrical microstimulation (ICMS) engage with cortical dynamics, revealing a new mechanism by which stimulation may evoke behavioral effects (see below!)