🧮 I finally spent some time learning what exactly Neural Tangent Kernel (NTK) is and went through some mathematical proof. Hopefully after reading this, you will not feel all the math behind NTK is that scaring, but rather, quite intuitive. lilianweng.github.io/posts/2022-09-…

Tips for ensuring reliable #blockchain #infrastructures: Testing infrastructures is important and different from testing application software. Here is how we do it at Sentio (full article linked below by Tianyi He and Kan) 🧵1/6

This work is a result of wonderful collaboration with Junwen Bai, @sidbrahma, Joshua Ainslie, Kenton Lee, Zhou Yanqi, Nan Du, Vincent Y. Zhao, Crick Wu, Bo Li, Yu Zhang, Ming-Wei Chang. We believe CoDA and CoLT5 highlight the potential of conditional computation for efficiency.

Excited about the simple and efficient technique developed from the team!

How do you turn a language model into a chatbot without any user interactions? We introduce LIMA: a LLaMa-based model fine-tuned on only 1,000 curated prompts and responses, which produces shockingly good responses. * No user data * No mode distillation * No RLHF

MoE Meets Instruction Tuning: A Winning Combination for Large Language Models [1/3] arxiv.org/pdf/2305.14705 Sheng Shen Zhou Yanqi Barret Zoph William Fedus Jason Wei Hyung Won Chung Shayne Longpre Xinyun Chen @tuvuumass Crick Wu Albert Webson Vincent Y. Zhao Denny Zhou

🤔️Why bother designing prompts to enable LLMs to self-improve? The preference data contains information on self-improvement. Thus, models can learn to self-improve implicitly! 📖Paper: arxiv.org/abs/2310.00898

Our team at Google DeepMind has a full-time Research Scientist position available at our Mountain View site. Minimum qualification: PhD in ML/NLP. Please email me with: your CV and Google Scholar link; a brief description of the impactful work you have done; and what you aim