In-context learning provides an LLM with a few examples to improve accuracy. But with long-context LLMs, we can now use *thousands* of examples in-context.

We find that this long-context ICL paradigm is surprisingly effective– and differs in behavior from short-context ICL! 🧵

Finally finished a new review paper: 'Machine learning for climate physics and simulations' Pedram Hassanzadeh We highlight the distinct yet complementary goals of ML: accelerating simulations vs learning physics. Share with us your favorite ML4climate papers! arxiv.org/abs/2404.13227

OpenDAC: an open dataset to train AI systems to help design compounds for the direct capture of CO2 from the air.

open-dac.github.io

I am excited to announce that our Categorical Deep Learning paper has been accepted to ICML!

arxiv.org/abs/2402.15332

Huge congratulations to the team Paul Lessard Andrew Dudzik Tamara von Glehn joao - mostly away Petar Veličković

and Symbolica for its first ICML paper!

I'm thrilled to announce that I’ve been accepted as a contributor for #GSoC2024 under The DeepChem Project. Looking forward to working with Bharath Ramsundar and Arun P for my project “Torch compile and PyTorch 2.2.0”.

Part of the reason I'm not that excited by LLM 'progress' is the rampant cheating. When you stress test an LLM out of its distribution it does quite poorly. This means LLM uses for science will be limited for now. Nothing wrong with partial progress, but the hype got very silly…

I have a suspicion that LLMs are actually quite far from maxing out even current benchmarks. If we rigorously disallow cheating and limit the training data as was common in older ML benchmarks, we would probably find a lot of scope for improvement. The problem is proprietary and…

📢 New preprint! Our latest work enables 'alchemy' - ∂[energy]/∂[element] in ML potentials like MACE. We model solid solutions and conduct alchemical free energy simulations. arxiv.org/abs/2404.10746
RGB Lab @ MIT #compchem #machinelearning

a fun couple of perspectives at stat.berkeley.edu/~songmei/resea…

arxiv.org/abs/2404.18444
arxiv.org/abs/2309.11420

We have a new blog post out on the Deep Forest Sciences blog covering our no-code Docking infrastructure. Our Prithvi platform enables biologists and chemists to use cutting edge AI and computational tools at scale without needing to write difficult distributed programs. Read…

There's a new bill, SB-1047 'Safe and Secure Innovation for Frontier Artificial Intelligence Models Act'.

I think it could do a great deal of harm to startups, American innovation, open source, and safety. So I've written a response to the authors: 🧵
answer.ai/posts/2024-04-…

DJI is the hardware version of Tiktok.
- Gets American operators addicted.
- Steals data, intelligence to PRC.
- Hollows out domestic drone production capacity.

Every American dollar going there is funding the Chinese military industrial complex.

The real threat to LLM chatbots isn't that adoption levels have stalled -- greater capabilities could reignite adoption. It's the fact that 1.5 years of massive investments haven't resulted in meaningfully better models, that would make a real difference in adoption potential.

Are there any deep dives about BYD in English? Seems like an extraordinary company worth learning from.

Probably a good idea for US entrepreneurs to study Chinese tech success stories in general. Maybe some good lessons for us there

Something like a Chips act could be used to bootstrap other manufacturing, including shipbuilding. Get the world's best companies to build in the US and use as a catalyst to revive the manufacturing base especially in areas where China has superior capabilities

Check out PySCF’s latest preprint in arXiv on the implementation of the GPU acceleration module, GPU4PySCF, which can deliver up to 30 times speedup over a 32-core CPU node, resulting in approximately 90% cost savings for most DFT tasks. 👀
arxiv.org/abs/2404.09452

If you’re a fan of DIY lab equipment, you’ll love the phenotype-o-mat, a customizable DIY 96-well plate imager from ArcadiaScience that you can now build for only $400 🧵 1/7
protocols.io/view/building-…

Our new work on generative models for chemical reactions: much faster inference with flow matching (OT path) training scheme, better leveraging our knowledge about the problem is a key to solve science problems! Check out the paper if you are interested!

Excellent thread about weird lobbying efforts that paint open source AI foundation models as anti-competitive (somehow!).
Laura Edelson is a Prof at Northeastern and the former Chief Technologist of the Antitrust Division of the Department of Justice.
She knows what she's…

Following up this take, LLMs may dramatically increase the importance of computer science and software engineering rather than reduce. Easy entry to computer programming is now open to anyone who can write structured English (and soon other languages). LLM agents for software…