Introducing PaperQA2, the first AI agent that conducts entire scientific literature reviews on its own. PaperQA2 is also the first agent to beat PhD and Postdoc-level biology researchers on multiple literature research tasks, as measured both by accuracy on objective benchmarks

Also, see our preprint for details, here: paper.wikicrow.ai 5/

We used PaperQA2 to extract claims from papers and then see if they're contradicted anywhere in literature. This task is time consuming for humans, but we were able to use this for hundreds of papers to look for trends in disagreement in fields, decades, and journals.

Wow: Just tried PaperQA, an open source AI-powered literature review whose research paper claims it achieves "superhuman synthesis of scientific knowledge" I tested it against papers I wrote and it seems like the real deal, putting together a good summary with accurate details.

It took us about 9 months of exploration to build agents that can do superhuman scientific literature summary and Q&A. Michael Skarlinski wrote up what failed and what was essential in an engineering blog post: futurehouse.org/research-annou…

Can you exchange frontier model compute for more accuracy in RAG? Yes. Check out our other experiments leading to the PaperQA2 design in our new engineering blog post: futurehouse.org/research-annou…

Another finding was how important it is to do multiple retrieval strategies. Using LLMs, you can get query expansion easily by having LLMs rewrite the question multiple ways. We also found exploiting sci lit metadata - i.e., citation graph - helped a lot 2/3