Really glad to see folks finally changing the RoPE hyperparams

There's our little thiel fellow! Why don't you come down and tell us about zero to one?

Every company on this list is bottlenecked by compute cost and speed. Grateful to be a Thiel Fellow and be working on Etched with Gavin Uberti Chris Zhu

The 21st century will be the most important century ever for humanity, thanks to the rapid advances in artificial intelligence. It makes no sense to sit on the sidelines in university.

I'm excited to join the latest class of Thiel Fellows

Black box access is all you need - arxiv.org/abs/2403.06634

Joe Heitzeberg OctoAI Perplexity Cloudflare Together AI Finally got around to this:
twitter.com/juberti/status…

Stable Diffusion 3 is a transformer 👀

Google open-sourced their Gemma models today, hyperparameters below. Both models have massive feed-forward hidden dimensions - almost every other model uses 3.5-4x the d_model (which would be 8192 and 12288). Not sure why the change.

Chain-of-Thought Reasoning Without Prompting

paper page: huggingface.co/papers/2402.10…

In enhancing the reasoning capabilities of large language models (LLMs), prior research primarily focuses on specific prompting techniques such as few-shot or zero-shot chain-of-thought (CoT)…