Emad Nathan Lambert processing all of CommonCrawl is about $20-50k [0], plus maybe 10-50k H100 if you wanna do GPU classification [1]. You can extract 1T tokens from PDFs for around $10k [2]. Major expenses are synth data, and verify which one of your approaches work [3]. -----------------------

Can not agree more!

MegaMath has been accepted to Conference on Language Modeling 2025🥳 Hoping you find our data useful!

🚀 Hello, Kimi K2! Open-Source Agentic Model! 🔹 1T total / 32B active MoE model 🔹 SOTA on SWE Bench Verified, Tau2 & AceBench among open models 🔹Strong in coding and agentic tasks 🐤 Multimodal & thought-mode not supported for now With Kimi K2, advanced agentic intelligence

Capable, Agentic, and Open-sourced. Kimi K2 excels in knowledge, math, and coding, and is optimized for complex tool use. See how it can analyze data, generate interactive webpages, and more. Explore what's possible and start building today!

🥳Happy to share that our paper "Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception" has been accepted by #ACL2025! We explore leveraging LLMs' internal states to improve their knowledge boundary perception from efficiency and risk perspectives.

Excited to share that our two papers have been accepted to #ICML2025! ICML Conference However, I can't be there in person due to visa issues. What a pity.🥲 Feel free to check out our poster, neither online nor offline in the Vancouver Convention Center. Programming Every Example:

MegaScience Pushing the Frontiers of Post-Training Datasets for Science Reasoning

🚨 New release: MegaScience The largest & highest-quality post-training dataset for scientific reasoning is now open-sourced (1.25M QA pairs)! 📈 Trained models outperform official Instruct baselines 🔬 Covers 7+ disciplines with university-level textbook-grade QA 📄 Paper:

When building MegaScience, we learned the hard way: 📈 Strong datasets need strong proxy models. Our data was too spicy 🌶️ for small models like Qwen2.5-1.5B & 3B—they just flopped. But once we tried Qwen3-14B and 30B… boom 💥, everything clicked. Kinda terrifying to think: if

🚀 In its first week, MegaScience ranks #4 on HuggingFace's Trending Datasets of the Week with 3.74k downloads! Thanks for the support — let’s keep pushing open science forward! 📷🌍

Next to Qwen3 of comparable size: Looks like gpt-oss is a wide (vs deep) model

Failing on 𝐥𝐚𝐫𝐠𝐞-𝐬𝐜𝐚𝐥𝐞 𝐑𝐋 with VeRL? ⚠️ Mixing inference backend (𝐯𝐋𝐋𝐌/𝐒𝐆𝐋𝐚𝐧𝐠) with training backends (𝐅𝐒𝐃𝐏/𝐌𝐞𝐠𝐚𝐭𝐫𝐨𝐧) 𝐬𝐞𝐜𝐫𝐞𝐭𝐥𝐲 𝐭𝐮𝐫𝐧𝐬 𝐲𝐨𝐮𝐫 𝐑𝐋 𝐢𝐧𝐭𝐨 𝐨𝐟𝐟-𝐩𝐨𝐥𝐢𝐜𝐲 — even if they share the same weights! 📉 Blog:

1. npx @​qwen-code/[email protected] 2. get 2000 free calls/day via Qwen Chat quick math: let's suppose avg agentic interaction ≈ 32k context 2000 × 32k ≈ 64 million tokens/day

🚀 OSWorld gets a major upgrade! OSWorld-Verified: 15 months community feedback → 300+ fixes (ambiguity, graders…), 50x faster eval through AWS parallelization More apple-to-apple comparison for reliable CUA evaluation ✨ 👇xlang.ai/blog/osworld-v…

Wow, this is really cool! This reserach answers this question: what if your computer-use AI was not a black box? OpenCUA: Open Foundations for Computer-Use Agents Researchers from HKU, Moonshot AI, and others present OpenCUA—a fully open-source framework for building and