Which modeling to choose for text-to-music generation? We run a head-to-head comparison to figure it out. Same data, same architecture - AR vs FM. 👇 If you care about fidelity, speed, control, or editing see this thread. 🔗huggingface.co/spaces/ortal16… 📄arxiv.org/abs/2506.08570 1/6

🚀 We're excited to announce our latest work: "Discrete Audio Tokens: More Than a Survey!" It presents a comprehensive survey and benchmark of audio tokenizers across speech, music, and general audio. preprint: arxiv.org/pdf/2506.10274 website: poonehmousavi.github.io/dates-website/

🚨 New paper alert! PAST: phonetic-acoustic speech tokenizer – just got accepted to Interspeech 2025 🎉 It learns phonetic + acoustic tokens jointly, with no SSL babysitter or external vocoder. 🔗pages.cs.huji.ac.il/adiyoss-lab/PA… 👇 If you’re into speech LMs, keep reading!

🎉Thrilled that our paper on "scaling analysis of interleaved speech-text LMs" was accepted to #CoLM2025 It gives room for optimism when scaling SpeechLMs *right* - with large TextLMs (in place of more data), interleaving, and synth training data💪

Join the inaugural 2025 Low-Resource Audio Codec (LRAC) Challenge on efficient neural speech codecs for resource-constrained devices! Learn more and sign up today! 👉 lrac.short.gy/call 📅 Registration is open; Challenge runs Aug. 1 – Sep. 30, 2025. #LRAC2025 #ICASSP2026

We trained a big neural network to generate code

New from Meta FAIR: Code World Model (CWM), a 32B-parameter research model designed to explore how world models can transform code generation and reasoning about code. We believe in advancing research in world modeling and are sharing CWM under a research license to help empower

We released CWM, a 32B LLM for code reasoning, agents, and world modeling research🚀 (pre/mid/post checkpoints, tech report, RL envs, inference stack): github.com/facebookresear…. I'm fortunate to lead Agentic RL and co-lead joint RL training, empowering CWM as a reasoning agent 🧵

Incredibly excited by this work, congrats Gabriel Synnaeve + AI at Meta codegen! 32b model that hits 65.8% on SWE-bench w/ TTS is incredible. A year ago that would've been unimaginable to me. Section 2 is a great read - resonates so much w/ what SWE-smith is trying to achieve in the open.

Kilian Lieret carlos Ofir Press Karthik Narasimhan Ludwig Schmidt Diyi Yang This table really puts things into perspective for me. SWE-smith has 250 repos (and 250 images, 1 per repo), and 26k OS trajectories (and counting). And these numbers were 10x more than any existing dataset at release. But CWM goes **another** order of magnitude larger 🤯

This is a very cool release. At the same time I'd like to see Meta be less cautious in their legal strategy around the release. The boundaries between the research & product communities are smaller than they've ever been, and noncommercial licenses really hamper innovation on

CWM makes me very bullish on Meta actually. It shows they (unsurprisingly) have all the fundamentals right and the headlines about them being lost are much exaggerated. Plus, it is independently really great program synthesis work and a genuine contribution!!

The craziest thing is that the blue bar is a smaller model than all the bars to the right by around 1 OOM... Makes me think it's rivaling the GPT-5s and Sonnet-4s if scaled up (2 OOMs bigger?)

1/ We released CWM, a 32B dense LLM for coding, agentic use, and, more importantly, to further World-Modeling research. To support this research, we release the pre-training, sft and rl model weights, along with inference code and the tech report. See:

So excited we finally get to share 🚀🔥Meta Code World Model! 🔥🚀 ✅ 32B dense open-weights LLM 💪 Strong coding skills 🧪 CWM enables a TON of cool research on world models for code generation! We would really like you to check it out, have some fun, and get inspired! 😊

Good analysis on Code World Model artificialintelligencemadesimple.com/p/the-illusion…

🚨 Attention aspiring PhD students 🚨 Meta / FAIR is looking for candidates for a joint academic/industry PhD! Keywords: AI for Math & Code. LLMs, RL, formal and informal reasoning. You will be co-advised by prof. Amaury Hayat from ecole des ponts and yours truly. You'll have

TL;DR: I made a Transformer that conditions its generation on latent variables. To do so an encoder Transformer only needs a source of randomness during generation, but then it needs an encoder for training, as a [conditional] VAE. 1/5

We present DyPE, a framework for ultra high resolution image generation. DyPE adjusts positional embeddings to evolve dynamically with the spectral progression of diffusion. This lets pre-trained DiTs create images with 16M+ pixels without retraining or extra inference cost. 🧵👇