Let t there be code reuse!

Hi Folks! Our newest episode of Portfolio Shorts is now live on YT! youtu.be/rqiwqjpE6RA Listen to Pawan Gupta talk about Fashinza which is a new-age supply chain and product development platform for the fashion industry🚀

Major AI labs are hitting the wall with larger models. True AI needs Continual Learning, Agency, and Adaptive behavior—not just bigger models and more data. My thoughts that Path to Prize (SuperIntelligence) lies beyond the comfortable confines of LLMs sushant-kumar.com/blog/path-to-p…

Pre-training SLMs seems to be accessible way into AI research. 1 day of H100-80GB pretrains ~1B SLM on roughly about 15B gpt2 tokens if properly optimised (was able to get upto ~200k tok/sec throughput) anyone know of SLMs (sub 1B) that has 50 MMLU (or in other words useful)?

One of best experiments to get hands dirty for building foundational models. ~30M model so cheap enough to pre-train. Added RoPE to GPT2 architecture. Raised PR! Pre-trained to a validation loss 1.224 in less than 1 hour H100 (40GB)! Great work, Om Alve! Toy MoE next?

Next week will be 🔥

Just got access to Claude Code! This one is quite good! Thanks Anthropic team!

awards.acm.org/about/2024-tur… Machines that learn from experience were explored by Alan Turing almost eighty years ago, which makes it particularly gratifying and humbling to receive an award in his name for reviving this essential but still nascent idea.

This lecture is a masterpiece from Richard Sutton that's moving the conversation forward on what the future with AI could look like. You can literally feel the clarity that comes when you have spent half-a-century thinking deeply about intelligence. youtube.com/watch?v=FLOL2f…

Looking forward to this … 🚀

For me, GPT-OSS was the much-awaited OpenAI architecture reveal since their detailed GPT-2 architecture! Some key observations: ✅ MoE GPT-2-style Transformer (36 / 24 layers) ✅ 128 / 32 experts with top-4 routing ⇒ only 5.1 B / 3.6 B active params ✅ RMSNorm ✅ Grouped Query

Soft deprecation of Codex?

Thanks Cursor team! This is awesome! It's going to be a exciting next few days on the house! 🚀

52% of the traffic on my personal AI blog (sushant-kumar.com) is from China! Seems like a good thing! 🚀

Exactly what I was searching the internet for today! Thanks Sebastian Raschka for always being on point!

I'm a simple person; I see a Rich Sutton lecture, I drop everything and watch!