Its here! Checkout our blog and paper on scaling synthetic data 1/n

Launch day! Take a look at the phenomenal insights into what production-grade synthetic data looks like - driven by Pratyush Maini and co

Today, we introduce BeyondWeb, our synthetic data generation approach which significantly outperforms all open synthetic data and is a key component of our curation pipeline. Many talk about “doing” synthetic data, but generating high-quality synthetic data at scale is extremely

The era of "The Era of Pretraining is Over" is over

Very excited to announce BeyondWeb, @datologyAI’s synthetic pretraining data generation paradigm. BeyondWeb is a rephrasing-based approach that substantially outperforms existing public synthetic pretraining data baselines, and is a core part of our curation pipeline.

Big day for the DatologyAI team! We introduce BeyondWeb, scaling synthetic data for trillion-scale pretraining! ✨ Collect more → curate better synthetic → win big ✨Not all synthetic data is created equal → doing it right pays off. ✨ With only targeted synthetic data

two paths for synthetic data

As we hit the limits of real web-scale data, DatologyAI's synthetic data shows how we can leverage the models we've already trained to squeeze even more value out of limited data! Huge props to Pratyush Maini for leading this work 👏🚀

OLMo 2 is SOTA web rewriter??

Great work by the DatologyAI team, enjoyed reading this

Thrilled to see BeyondWeb launched 🚀 Phenomenal insights and a huge step forward for scaling high-quality synthetic data to trillions of tokens. Amazing work by Pratyush Maini and the DatologyAI team - super excited to be learning from you all!

Pratyush Maini and DatologyAI make synthetic data seem easy, but it's really just how good they are

For years, researchers have known that synthetic data is valuable; but not all synthetic data is created equally. Generating high quality synthetic data not only requires specific research expertise it also requires you to deeply understand [your] data. These key learnings are a

its a pleasure to show up to work everyday and learn from synthetic data gurus Pratyush Maini , Vineeth and team -- this blogpost is a work of art 👌👌

Finalizing the magic seed for synthetic data generation while the data dawg Ricardo Monti showers his blessings in the background.

After months of development, we finally share with the world some hard-earned science behind synthetic data. DatologyAI presents BeyondWeb — a SOTA approach showing how thoughtful synthetic data design can beat strong baselines.

The last two days have been a whirlwind, and I haven’t had a chance to read this end to end - though I did see an early draft - let alone comment. I’m one of the few people outside DatologyAI fortunate enough to have seen these results firsthand, and everyone can experience