Language Model Cascades arxiv.org/abs/2207.10342 Good paper and all the references (chain-of-thought, scratchpad, bootstrapping, verifiers, tool-use, retrievals, etc...). There's a quickly growing stack around/above a single large language model, expanding their reasoning power

Is someone aware of a language model experiment where you keep all the 2022 goodies/data, except swap a Transformer for an LSTM? I expect a gap should exist and is worth thinking about more closely, e.g. from the perspective of being both 1) expressive and 2) SGD optimizable.

Doing some cleaning around the house, stumbled on a notebook filled with failed proofs for what would eventually become our MICo distance for MDPs. It took about a year of banging our heads with failed ideas until we stumbled on the one in our paper, which works really well!

[5/8] Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Jin-Hwa Kim, Byoung-Tak Zhang. SelecMix: Debiased Learning by Contradicting-pair Sampling.

This neural network architecture that was showcased at the Tesla AI day is a perfect example of Deep Learning at its finest. Mix and match all the greatest innovations to do something drastic and super ambitious. Congrats!

World Model is a causal predictor Decision Transformer is an anti-causal predictor Hindsight Experience Replay is the Jedi mind trick to flip the causality

Yet people can learn without simulators, and even without the Internet. Even what we call "imitation" in robotics is different from how people imitate. As for data, driving looks different from robotics b/c we don't yet have lots of robots, but we will: sergeylevine.substack.com/p/self-improvi…

Today in nature: #AlphaTensor, an AI system for discovering novel, efficient, and exact algorithms for matrix multiplication - a building block of modern computations. AlphaTensor finds faster algorithms for many matrix sizes: dpmd.ai/dm-alpha-tensor & dpmd.ai/nature-alpha-t… 1/

We trained a transformer called VIMA that ingests *multimodal* prompt and outputs controls for a robot arm. A single agent is able to solve visual goal, one-shot imitation from video, novel concept grounding, visual constraint, etc. Strong scaling with model capacity and data!🧵

In our new work - Algorithm Distillation - we show that transformers can improve themselves autonomously through trial and error without ever updating their weights. No prompting, no finetuning. A single transformer collects its own data and maximizes rewards on new tasks. 1/N

"SelecMix: Debiased Learning by Contradicting-pair Sampling" Inwoo Hwang · Sangjun Lee · Yunhyeok Kwak · Seong Joon Oh · Damien Teney · Jin-Hwa Kim* · Byoung-Tak Zhang* Hall J #426 at 4 PM neurips.cc/virtual/2022/p…

This is natural given the vast quantity of data suitable for SSL. Nevertheless, I'm predicting a bit of a comeback for RL as we try to shape and refine these systems for various applications.

We train Transformers to encode algorithms in their weights, such as sorting, counting, and balancing parentheses from lots of data. I never thought we may also go in the *reverse* direction: *compile* Transformer weights directly from explicit code! Cool paper @DeepMind: 1/🧵

yay the ability to share ChatGPT conversations is now rolling out. I can share a few favorites. E.g. GPT-4 is great at generating Anki flash cards, helping you to memorize any document. Example: chat.openai.com/share/eef34fe5… Easy to then import in Anki: apps.ankiweb.net

We're done! A historic milestone for neuroscience brought to you by the FlyWire Consortium. Don't take my word for it. See the glory of the fly brain for yourself at flywire.ai/gallery

Paper rejected from #NeurIPS2023. Frustrated initially as I felt some reviewers focused on irrelevant issues. But it's also a chance to improve. I'll work on better writing and clearer articulation for the next round, and try to keep a positive and humble attitude! 📝🌱 #PhDLife

A large part of my research has been inspired by his theory of System 1 & 2. RIP Daniel Kahneman.

[1] Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction (Oral) paper: arxiv.org/abs/2406.00614 We propose state-conditioned action abstraction that effectively reduces the search space of MCTS under vast combinatorial action space.

[1] Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning Tuesday, 13:30-15:00 paper: arxiv.org/abs/2406.03234 We propose a principled approach to discovering fine-grained causal relationships with identifiability guarantees.