
Krunoslav Lehman Pavasovic
@krunolehman
PhD in Generative AI @Meta & @ENS_ULM. Previously at @Inria, @ETH, @UniOfOxford, @CERN.
ID: 1576609018054905856
02-10-2022 16:24:22
10 Tweet
74 Followers
253 Following

Neat paper on classifier-free guidance by K. Pavasovic, J. Verbeek, Giulio Biroli & M. Mezard: arxiv.org/abs/2502.07849โฆ


๐จ Your RL only improves ๐ฝ๐ฎ๐๐@๐ญ, not ๐ฝ๐ฎ๐๐@๐ธ? ๐จ Thatโs not a bug โ itโs a ๐ณ๐ฒ๐ฎ๐๐๐ฟ๐ฒ ๐ผ๐ณ ๐๐ต๐ฒ ๐ผ๐ฏ๐ท๐ฒ๐ฐ๐๐ถ๐๐ฒ youโre optimizing. You get what you optimize for. If you want better pass@k, you need to optimize for pass@k at training time. ๐งต How?



We present an Autoregressive U-Net that incorporates tokenization inside the model, pooling raw bytes into words then word-groups. AU-Net focuses most of its compute on building latent vectors that correspond to larger units of meaning. Joint work with Badr Youbi Idrissi 1/8
