Imitation directly from images, inverse RL with with transferable rewards, and better GANs can all be enabled by adding an information bottleneck into the discriminator!

Learn concrete tips on how to successfully build useful #MachineLearning projects from @EmmanuelAmeisen, Head of AI @InsightDataSci and Adam Coates, operating partner Khosla Ventures. blog.insightdatascience.com/how-to-deliver…

Iterative development is one of the most important disciplines I try to share with every ML team I work with. @EmmanuelAmeisen from @InsightDataSci and I teamed up to share the process and tips for new ML Engineers! blog.insightdatascience.com/how-to-deliver…

CACTUs: an unsupervised learning algorithm that learns to learn tasks constructed from unlabeled data. Leads to significantly more effective downstream learning & enables few-shot learning *without* labeled meta-learning datasets arxiv.org/abs/1810.02334 w/ Kyle Hsu, Sergey Levine

I appear to be using GMail's "sentence auto-complete" feature to actively avoid low-information-content sentences. 🤔

The Extinction Illusion: The left side and the right side have the same number of black dots journals.sagepub.com/doi/abs/10.106…

So many papers applying deep learning to theoretical and experimental physics! Fascinating. physicsml.github.io/pages/papers.h…

This is being reported as a problem with machine learning, but there's another way of looking at it: The algorithm exposed bias in their *existing* hiring practices

This is one of the things about bias in ML that I fear the most. Many companies will build these systems innocently and have *no idea* that the algorithm has found a way to discriminate unfairly. *Everyone* in company has to stay vigilant, not just ML team.

I've spent most of 2018 training models that could barely fit 1-4 samples/GPU. But SGD usually needs more than few samples/batch for decent results. I wrote a post gathering practical tips I use, from simple tricks to multi-GPU code & distributed setups: medium.com/huggingface/tr…

Got the "do I use an RNN or CNN for sequence modeling" blues? Use a TrellisNet, an architecture that connects these two worlds, and works better than either! New paper with Shaojie Bai and Vladlen Koltun. Paper: arxiv.org/abs/1810.06682 Code: github.com/locuslab/trell…

I have put together some thoughts about the article here: bit.ly/2AxgTvE. I am concerned that the story in the Economist not only displaces my own narrative but also sets unrealistic expectations for anyone setting out in the field.

Our field needs more diversity. There must be more people like me who feel welcome and who are given the tools to succeed. However, part of preparing people to succeed is to be candid about how challenging it can be and how many failures there are along the way.

It turns out that DL's ability to make great predictions of just about anything is a still underestimated superpower. "Curve fitting" is doing some nifty things these days. 👇👇👇

WaveGlow: A non-autoregressive generative model for speech synthesis. Our unoptimized PyTorch inverts mel-spectrograms at 500 kHz on a V100 GPU, and is easy to train. Paper: arxiv.org/abs/1811.00002 Samples: nv-adlr.github.io/WaveGlow Work with Ryan Prenger and Rafael Valle

Bigger models are better models so we built GPipe to enable training of large models. Results: 84.3% on ImageNet with AmoebaNet (big jump from other state-of-art models) and 99% on CIFAR-10 with transfer learning. Link: arxiv.org/abs/1811.06965

Ankesh Anand Edward Grefenstette Yann LeCun I posted the slides for my model-based RL tutorial today here: people.eecs.berkeley.edu/~cbfinn/_files… Content is similar to the DRL bootcamp video 1 yr ago, but updated.

New blog post: "A Recipe for Training Neural Networks" karpathy.github.io/2019/04/25/rec… a collection of attempted advice for training neural nets with a focus on how to structure that process over time

All materials of Berkeley AI Research Deep Unsupervised Learning now up: sites.google.com/view/berkeley-… Great semester w/Peter Chen,@Aravind7694,Jonathan Ho, and guest inst. Alec Radford,Ilya Sutskever,A Efros,Aäron van den Oord Covers: AR / PixelCNN, Flow models, VAE, GAN, self-supervised learning, etc...