btw if you as a developer have not been paying attention to the tech earnings season, look out.

basically market is bifurcating ai winners and losers hard rn. if you’re not an ai arms dealer or ai consumer company we basically just flipped to a bear market this week

currently

wow characterizing teams as an MVP feature succinctly puts the strong opinion where it hurts most. love.

🆕 Andrej Karpathy has started following @smol_ai

Excited to announce Devendra Chaplot as a speaker! He's an AI Research Scientist Mistral AI and will provide an inside look at the building blocks and training strategies that power Mistral's LLMs.

Join him June 25 - 27! 🎟️ link in next post

quite incredible to see the goalposts of long context move in the past 1 year.

May 2023: asking Jonathan Frankle and Abhi Venigalla about their 65k+ 'Llongboi' model

May 2024: Mark Huang casually extending Llama 3 to >1m tokens with ~perfect NIAH and mainstream ai engineers

🆕 pod: How to train a Million Context LLM!

Yann LeCun says we should publish, or perish. We asked Mark Huang to spill ALL the beans on how his team extended Llama-3 to 1M+ context with ~perfect Greg Kamradt NIAH evals!

latent.space/p/gradient

Also covering:

- A Brief

periodic reminder that i do maintain a list of technically focused ai podcasts and newsletters here

github.com/swyxio/ai-note…

obviously its hard to cover math and images on voice format but the people who #learninpublic are heroes for helping the rest of us keep up

Heads up comparison, @ElevenLabsio vs Cartesia

Cartesia almost completes before Elevenlabs starts lol

realtime SSMs are going to be -wild-

congrats Karan Goel Albert Gu

AINews: 29 May 2024

What if you KNEW that we may soon have models can that continuously process and reason over text/audio/video with a TRILLION token 'context window'?

Real time? On device?

thanks to Cartesia, Karan Goel, Albert Gu

🏆 swyx 🔜 SFO! and Alessio Fanelli went to ICLR 2024 and chose 14 outstanding #ML and #AI papers to highlight from the conference. Check out their list! latent.space/p/iclr-2024-re…

#MachineLearning #ArtificialIntelligence #ML Ops #LLMOps #GenAI #GenerativeAI #DataEngineer #DataScience

Thrilled to unveil Maven AGI, a reimagined customer support experience platform emerging with $28M in funding from Lux Capital M13 E14 Fund & OpenAI Stripe HubSpot execs! 🪄

Read more with their OpenAI case study 🧵
shorturl.at/ZXX3U

🆕 ICLR 2024: Best Papers (Part 1)

We present our selections of outstanding papers and talks thematically introducing topics for AI Engineers to track:

Section A: ImageGen, Compression, Adversarial Attacks

- Auto-encoding Variational Bayes (Durk Kingma)
- Würstchen: An Efficient

# Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 ✨

The GPT-2 (124M) is the smallest model in the GPT-2 series released by OpenAI in 2019, and is actually quite accessible today, even for the GPU poor. For example, with llm.c you can now reproduce this model on one 8X

These are the problems we believe matter most for the next year of ai-programming

anysphere.inc/blog/problems-…

My favorite is next action prediction, the limit of what's possible with low-latency models

(we press tab 11 times and all other keys 3 times to make these changes)
[1/7]

ICLR episode part 1 just shipped! technically still in time for the long weekend haha

on a meta level it’s been great studying all the big ML conferences in prep for AI Engineer next month. Lots of good ideas for running >5000 person affairs with something for everyone. Any