We just released the largest open-source diffusion language model (RND1). RND1 is important to me on a personal level: it symbolizes our commitment to open-source exploration of radically different designs for AI at scale — training objectives, architectures, domains. There is

Excited to share Phalanx, our new layer for sequence modeling! Each block communicates with its neighbor, like the shield cover of a neighboring hoplite. Phalanx can replace sliding window attention and trains faster than optimized baselines while maintaining quality.