For the last few years, one of the staples of my research has been visualizing individual neurons in vision models. But that's only a partial picture -- neurons work together. Activation Atlases are a way to explore the space neurons jointly represent. distill.pub/2019/activatio…

Blogpost on our new theory for word2vec-like representation learning methods for images, text, etc. Explains why representation do well on previously unseen classification tasks offconvex.org/2019/03/19/CUR… Relevant to meta learning, transfer learning? Paper arxiv.org/abs/1902.09229

Other fundamental tool for non-smooth optim: subdifferential of f at x = set of all slopes of affine minorants of f which are exact at x. Coincides with {∇f(x)} if f cvx & differentiable. Leads to 1st order optimality condition for *non-smooth* cvx fcts Fabian Pedregosa Pierre Ablin

PyTorch BigGraph: a distributed system for learning large graph embeddings - up to billions of entities and trillions of edges - Sharding and Negative Sampling - WikiData embeddings (78 mil entities, 4131 relations) - Blog: ai.facebook.com/blog/open-sour… - Code: github.com/facebookresear…

Coming events, where DataPink will be present: - April 18 #AINIGHT (booth) - May 22-23 Joint Urban Remote Sensing Event (booth) - June 11-12 AI PARIS (booth) - June 13 OW2 → https://fosstodon.org/@ow2 (talk) Hope to see you there !

A walkthrough of the PyTorch Internals by core developer Edward Z. Yang . It's a great resource if you want to contribute to PyTorch. blog.ezyang.com/2019/05/pytorc…

#TheroreticalNeuroscience in 2 links: lps.ens.fr/~risc/CA6/lect… gatsby.ucl.ac.uk/~lmate/biblio/…

Great news! We've open-sourced HighRes-net for Multi-Frame Super-Resolution by Recursive Fusion. github.com/ElementAI/High… Our #PyTorch implementation that topped AdvancedConceptsTeam's competition. We're hoping this will help researchers working with satellite imagery #AIforGood @element_ai

Next week is #ARD19, a remote sensing data interoperability workshop to be held in Menlo Park. In anticipation of the event, our senior director of analytics engineering tackles essential ingredients for ecosystem-wide geospatial analytics capability. bit.ly/2SQZ56a

For the first time in the series, #SpaceNet round 5 will use a "mystery city" for final scoring! Winning algos will have to work not only on the training cities, but on a new, different geography. More details: medium.com/the-downlinq/s…

Optimal Transport generalizes sorting to spaces of dimension D > 1. GeomLoss implements a smooth Quicksort-like solver, packaged as a PyTorch layer. Keeps a linear memory usage + x10 to x1,000 speedup compared with the Auction and Sinkhorn algorithms. bit.ly/2PfbliN

#ComputerVision ecosystem for GeoSpatial Imagery, at scale: @RoboSatPink datapink.com/presentations/… #NLP: From Text2Map, a state of art. datapink.com/presentations/… Both talks FOSS4G 2024 | https://fosstodon.org/@foss4g, on how to use #DeepLearning to extract valuable patterns from #GeoSpatial (open)data.

I'm so excited about our current projects on energy-efficient NLP! Distilled models are very complementary to larger models. Training cost make headlines but as large-scale models reach production, inference time will likely account for most of a model's total environmental cost

C'est la rentrée ! Le #meetup #PAU #MachineLearning ouvre ses portes le mercredi 11 Septembre à 18h30 Technopole Hélioparc . Au menu #ComputerVision, #DeepTech et #NLP avec Olivier Courtin from DataPink Inscription ICI : meetup.com/fr-FR/Meetup-M…

The paper that introduced Batch Norm arxiv.org/abs/1502.03167 combines clear intuition with compelling experiments (14x speedup on ImageNet!!) So why has 'internal covariate shift' remained controversial to this day? Thread 👇

PyTorch 1.3 includes support for model deployment to mobile devices, quantization, & front-end improvements, like the ability to name tensors. New tools & libraries are also launching for improved model interpretability & multimodal development. Read more: ai.facebook.com/blog/pytorch-1…

DataPink will be PARIS SPACE WEEK 2020 Plan to see us on our booth, for neat-EO.pink demo #AI4EO

WTF? We brutally dismember BERT and replace all his organs? 👉🏻 Check our latest work: arxiv.org/abs/2002.02925 ⛵️ BERT-of-Theseus ⛵️ *NEW MODEL COMPRESSION METHOD* *ONE* loss + *ONE* hyperparameter + *NO* external data = GREAT PERFORMANCE with a Hugging Face -compatible weights