Tom Sydney Kerckhove Dual blackwell 6000 pro + threadripper 7000/epyc turin/genoa/intel emerald rapids platform

implement difficult (but interesting) projects. stuff ~20% above your current ability it raises your ceiling for solving hard problems, it makes you learn new stuff, and it makes you very efficient at easier problems repeat over a long period of time x.com/MaxMynter/stat…

4o image gen made a banner so good I changed back to my old pfp to go with it (yes it's gura themed)

How much degrees should I tolerate maximum on the A100’s ? I don’t want to damage them

Anyi Rao OpenAI I’d love to see a recording of this talk

managed to write a flash attention 2 which is slower than my flash attention 1 ....

btw RWKV PENG Bo proposed this (2-token hard-coded causal convolution) in 2020-08-20 (YYYY-MM-DD). zhuanlan.zhihu.com/p/191393788

someone should probably retry all those late 2010s deep RL ideas to see if they work on LLMs

FYI the GPU Glossary is open source now! Have a few tasks to tackle listed in the Issues. github.com/modal-labs/gpu…

Apparently I can’t ask more than a 1000W to the server 1400W psu without the fan going supersonic jet mode

Introducing pi-quant, the Prime Intellect Fast Quantization Library. Hand-tuned, parallel CPU per-tensor quantization kernels, over 2x faster than PyTorch on all tested hardware. Optimized for various CPU architectures.

my pop impl works so well it unlearns the environment

Maybe we shouldn't force thinking models to think in English or any specific language. It's your skill issue if you don't speak 40+ languages.