🚨Ilya Sutskever finally confirmed > scaling LLMs at the pre-training stage plateaued > the compute is scaling but data isn’t and new or synthetic data isn’t moving the needle What’s next > same as human brain, stopped growing in size but humanity kept advancing, the agents and

Ilya Sutskever says that data is the fossil fuel of AI and "we have achieved peak data" because "we have but one internet", so the age of pre-training AI on bigger and bigger compute clusters "will unquestionably end"

The original video from the talk by Ilya Sutskever at the Neural Information Processing Systems (December 10 - 15, 2024, Vancouver)

Good read anthropic.com/research/build…

Video of the day .. youtube.com/watch?v=knDDGY…

Friday afternoon snack from BG2. Greatest hits from 2024 👊💥 Brad Gerstner Bill Gurley

More than 1500 years before Silicon Valley debated AI consciousness, ancient Indian materialists (Cārvākas) already proposed that consciousness emerges from material elements alone! 1/

New 3h31m video on YouTube: "Deep Dive into LLMs like ChatGPT" This is a general audience deep dive into the Large Language Model (LLM) AI technology that powers ChatGPT and related products. It is covers the full training stack of how the models are developed, along with mental

Wow! This is just from one lab. The ROI on government grants is astronomical, especially when you consider that most of the value to society from basic science goes way beyond stock market valuations. Defunding one America's greatest successes is insanely bad policy.

When working with LLMs I am used to starting "New Conversation" for each request. But there is also the polar opposite approach of keeping one giant conversation going forever. The standard approach can still choose to use a Memory tool to write things down in between

An agent-first world is around the corner — where more actions will likely be taken by AI than by humans when building software. Zeno Rocha, cofounder of Resend, and Yoko Li say this reality is shaping how developers build, and changing what user experience really means. Great

Case and point

Andrej Karpathy Exactly the same pain. With many years of education in CS, it feels like I had no skills to ship complete end-to-end apps. Here's a course where we attempt to solve this pain: ship-ai.csail.mit.edu Its our first attempts and covers the basics, but still there's so much more.

👿 MCP is all fun, until you add this one malicious MCP server and forget about it. We have discovered a critical flaw in the widely-used Model Context Protocol (MCP) that enables a new form of LLM attack we term 'Tool Poisoning'. Leaks SSH key, API keys, etc. Details below 👇

x.com/i/article/1909…

Test the idea 💡

Do you think that ChatGPT can reason? youtu.be/y1WnHpedi2A?si… via YouTube Wow worth your hour and half time !!

It’s happening. AI models are finding zero-day vulnerabilities. This blog post is a must-read and marks a new era for cybersecurity.

Some of the links: - My slides as keynote: drive.google.com/file/d/1a0h1mk… - Software 2.0 blog post from 2017 karpathy.medium.com/software-2-0-a… - How LLMs flip the script on technology diffusion karpathy.bearblog.dev/power-to-the-p… - Vibe coding MenuGen (retrospective) karpathy.bearblog.dev/vibe-coding-me…

Lovable just crossed $100M ARR in 8 months. Faster than OpenAI, Cursor, Wiz, and every other software company in history. Today we're launching a game-changing update, it reduces the error rates by 91%. Introducing Lovable Agent: