been playing with 2 amd mi50, okay performance with sglang. qwen3-8B ~60t/s gen with tp2 small ctx, (~900GB/s reached bw), 50t/s ctx 4k qwen3-4b single gpu gets ~80t/s (~600GB/s bw), individual bws: -up+g matmul kernels reach ~850GB/s -down mm 800GB/s -qkv 870GB/s -o 800GB/s

feel like clipping could be used everywhere to make the activations quantization friendly

twitter perception of openai

zoom out

the curse of causal masks

interesting pair of papers yday bytedance chads with ultramemv2 arxiv.org/abs/2508.18756 Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks arxiv.org/abs/2508.18672

lots of tts activity

i think i’ll make a sign

with language diffusion models research slowly catching, my biggest hunch is that param heavy encoder-small weights, heavy flops decoder would greatly fit current consumer hw. waiting eerily for what neurips.cc/virtual/2025/p… has to show us

tts on neural engine, not real time yet (on m1 air), but we'll get there

new deepseek ocr model feels fun

deepseek taking the encoder decoder route

hopefully apple will open up their npu, missing out a lot and gap will only widen faster