Exciting to see the response to our work on Lagrangian Neural Networks: reddit.com/r/singularity/… Does our work suggest that AI can develop physics without any form of prior knowledge?

It's a real shame that ICML has decided to automatically reject accepted papers if no author can attend ICML. A top conference paper is a significant boost to early career researchers, exactly the people least likely to be able to afford to go to a conference in Vancouver.

Superposition and neural scaling laws are the two amazing phenomena in language models. Our new work shows that they are the two sides of the same coin! In practice, one can control scaling by controlling superposition via a “negative” weight decay, which is kinda crazy :-)

🚀 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗜 𝗶𝘀 𝘁𝗵𝗲 𝗡𝗲𝘅𝘁 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗟𝗲𝗮𝗽 𝗳𝗼𝗿 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗗𝗿𝗶𝘃𝗶𝗻𝗴—𝗛𝗲𝗿𝗲’𝘀 𝘁𝗵𝗲 𝗛𝗶𝘁𝗰𝗵𝗵𝗶𝗸𝗲𝗿'𝘀 𝗚𝘂𝗶𝗱𝗲 𝗬𝗼𝘂’𝘃𝗲 𝗕𝗲𝗲𝗻 𝗪𝗮𝗶𝘁𝗶𝗻𝗴 𝗙𝗼𝗿! 🚀 We're thrilled to share our most comprehensive, 128-page

Today, the most competent AI systems in almost *any* domain (math, coding, etc.) are broadly knowledgeable across almost *every* domain. Does it have to be this way, or can we create truly narrow AI systems? In a new preprint, we explore some questions relevant to this goal...

Check out our new paper on using AI to make progress on the Millennium problem of fluid blowup!