🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

want to clarify some common misunderstandings - this paper is about elicitation, not self-improvement. - we're not adding new skills --- humans typically can't teach models anything superhuman during post-training. - we are most surprised by the reward modeling results. Unlike

Join DatologyAI if you have conviction on #4

A good language model should say “I don’t know” by reasoning about the limits of its knowledge. Our new work AbstentionBench carefully measures this overlooked skill in leading models in an open-codebase others can build on! We find frontier reasoning degrades models’ ability to

Welcome Vineeth! We're hiring across the board to build the best data engine for AI. Come join us! jobs.ashbyhq.com/datologyai

Our customers needed a better base model <10B parameters. We spent the last 5 months building one. I'm delighted to share a preview of our first Arcee Foundation Model: AFM-4.5B-Preview.

We teamed up with DatologyAI to build what we believe is the strongest pretraining corpus in the world—and I truly think we nailed it. Their team was absolutely key to the model’s success. We started with ~23T tokens of high-quality data and distilled it down to 6.58T through

It turns out LLM data is more like oil than coal, if you refine it properly. Congratulations to the contributors of the many researcher-years of work!

Congratulations to our friends and partners Arcee.ai on the release of AFM-4.5B! With data powered by DatologyAI, this model outperforms Gemma3-4B and is competitive with Qwen3-4B despite being trained on a fraction of the data.

This trend will only continue. Training your own model doesn't need to cost 10s of millions, especially in specialized domains. Better data is a compute multiplier, and DatologyAI's mission is to make this easy, massively reducing the cost and difficulty of training.

Data powered by DatologyAI 😎

Training costs have definitely been going down, but I think there's still a massive barrier when it comes to data, especially if you want to train on proprietary datasets. DatologyAI, we are razor focused on changing that.

Im going to have to ask you to STOP LOOKING AT NEWEST ATTENTION VARIANTs and LOOK AGAIN AT YOUR FUCKING DATASET

This is why automating data curation is not only necessary for scalability reasons (a human obviously can't curate trillions of tokens), but also because humans aren't actually *good* at assessing data quality.

It depends on how much you know about what you're using your model for. You want your data to be as similar to your test distribution as possible. In practice, benchmarks are an incomplete description of your true test distribution, so you want to hedge diversity vs.

🌞 We're excited to share our "Summer of Data Seminar" series at DatologyAI! We're hosting weekly sessions with brilliant researchers diving deep into pretraining, data curation, and everything that makes datasets tick. Are you data-obsessed yet? 🤓 Thread 👇

STOP LOOKING AT SUBQUADRATIC ATTENTION PAPERS and GET BETTER DATA

We’re excited to introduce Chai-2, a major breakthrough in molecular design. Chai-2 enables zero-shot antibody discovery in a 24-well plate, exceeding previous SOTA by >100x. Thread👇

the other bitter lesson: data >>> code/arch