📢 Introducing IfQA - the first large-scale open-domain question answering (ODQA) dataset centered around counterfactual reasoning. Together with Meng Jiang Aristo Team at AI2! Paper link: arxiv.org/abs/2305.14010

📢 Introducing ReFeed: a novel plug-and-play approach to enhance the factuality of large language models via retrieval feedback! Together with Meng Jiang Zhihan Zhang Zhenwen Liang Aristo Team at AI2 Read more: arxiv.org/abs/2305.14002

Introducing 𝗥𝗘𝗙𝗟𝗘𝗫: What does my LLM believe?🧐We show that we can add a 𝗿𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗹𝗮𝘆𝗲𝗿 to an LLM to materialize its "belief graph", repair inconsistencies & produce reasoning chains drawn from a now-consistent system of beliefs! arxiv.org/abs/2305.14250 #NLProc

New paper: "Attentiveness to Answer Choices Doesn’t Always Entail High QA Accuracy" 📊💬 sarahwie.github.io/attentiveness.… Something I've been thinking a lot about recently is the relationship between distributions over vocabularies produced by language models and the various ways... 1/5

ICYMI Ben Brubaker wrote an eloquent Quanta article✍️covering our ICLR-2024 paper (w/ William Merrill 🚂ACL) on how the expressive power of transformers changes with the length of CoT! Recently updated paper📜at arxiv.org/abs/2310.07923

Turns out SSMs like S4 and S6 don't quite get the best of both worlds -- sequential and parallel -- and struggle to track state just like Transformers. Excited to share the "Illusion of State" paper w/ William Merrill 🚂ACL, Jackson Petty ! arxiv.org/abs/2404.08819

Happy to share that this work will appear at NAACL-2024! Check out our recently updated version on arXiv at arxiv.org/abs/2311.09519

Excited to share AppWorld, our challenging new interactive coding environment and benchmark to push AI agents further! Super easy to use (`pip install...`), reliable, reproducible, realistic. Congratulations to Harsh Trivedi for the huge effort!!

🥳The BIGGEST congratulations for our teams' recognition at #ACL2024! OLMo received the Best Theme Paper, Dolma + AppWorld received the Best Resource Paper, and "Political Compass or Spinning Arrow?" was honored with an Outstanding Paper Award.

Thrilled for AppWorld, and especially Harsh Trivedi, to receive this recognition at ACL-2024!

Excited to share this new benchmark for LLM-based coding agents, for a "super"-useful problem! Kudos to Ben Bogin tusharkhot and colleagues!