I've been working on a new product: MakeDistribution.com!

It's the best solution to creating a subjective probability distribution, i.e. one that reflects human judgement rather than being fit to data.

Sounds simple, but most existing solutions have important flaws (1/15)

Come work with me! You might be a great fit for this role if you have an operations mindset, strong communications skills, are service-minded and organized, and care deeply about Epoch’s mission.

Who are some of the top historians of the field of AI?

Cool to see our replication of Chinchilla amongst the top ML papers of the week in what was a packed week for AI.

This is how you would train compute-optimal models to match Llama3 using our updated Chinchilla scaling law.

Models are clearly being overtrained 5x to 10x further than is *pre-training* compute optimal.

Algorithmic progress is faster than hardware progress.

arxiv.org/abs/2403.05812

I'm thrilled to see that our work has apparently unified the Chinchilla scaling laws. It's great to hear that they're making the data open source!

great to see a major paper like Chinchilla be updated and releasing data

data forensics 101

we are in the csi miami 'enhance' era of reproducibility research

I don't know and can't assess the impact of the results on the topic of scaling laws, but the reproducibility effort is remarkable! We need more work like this, in many fields of CS.

tl;dr: the parametric Chinchilla scaling law appears to have been poorly fit, undermining any analysis that relied on its exact fitted values. We fit the same scaling law to a reconstruction of their data, getting different and IMO better results.

Exhibit #49864 on the absurd lengths reproducibility studies must go in the era of proprietary LLMs 😭

Intriguing and potentially important work!