Portrait of hound hunting in hills above Gilroy, c. 2024, by AMD 7900 XTX

Anyone able to read the tealeaves as to wtf is going on here?

Great work Black Forest Labs!

Finally, Hermes 3 in 405B!

🔥 PyTorch 2.4.0 on AMD is *huge* improvement! E.g., for same setup as plot below, perf is now at ~245,000 toks/sec! - compile() works; - flash attention works; - perf tuning works. It is now faster than the hipified llm.c! 🎉 Thank you AMD, PyTorch & Soumith Chintala!

"Apollo and the iPhone 16 Pro Max"

Pierre Vannier - AI Engineer // 🇪🇺eu/acc Marc Andreessen 🇺🇸 Well Marc Andreessen 🇺🇸 did say Apollo... but here's "Aphrodite with the iPhone of the gods"

Wow, I just assumed that Zuck had created his own masterpiece outta chicken wire and bondo... turns out it is the work of a real artist

Finally got around to trying out llm.s on MI300x.. the code I had tuned on MI250x gets decent perf straight outta the gate. But wtf is going on with PyTorch perf on MI300x? Tried 2.4 & nightly, with rocm 6.1&6.2.. using full autotuning, flash attention etc.. all are so slow

Awesome! Also for local training, this would enable training across a bunch of machines without expensive interconnect :)

Hitting ~3.2M toks/sec on MI300x for tiny llama3 training. Would love to see @pytorch training on MI300x get some massive improvements so I don't have to write my own kernels!

Was the model trying to upload itself? lol