Are large language models superhuman chemists?

We have been wondering about this, too and developed the first automated and comprehensive benchmark for evaluating large language models in chemistry and let LLMs compete with human chemists 👩🏼‍🔬

📝 arxiv.org/abs/2404.01475