One paper, lead by kstechly (w/ Matthew Marquez), evaluated the claims over a suite of graph coloring problems. The setup allows for GPT4 guessing a valid coloring in stand alone and self-critiquing modes. There is an external sound verifier outside the self-critiquing loop. 2/

📢 Check out these posters on LLM Self-Critiquing (in)abilities in reasoning and planning tasks, being presented at the #NeurIPS2023 "Foundation Models for Decision Making" workshop today (12/15) by yochanites Karthik Valmeekam and kstechly in Hall E2.

📢 "On the self-verification limitations of LLMs in Reasoning and Planning Tasks" arxiv.org/abs/2402.08115 (lead by Karthik Valmeekam and kstechly) 👇 Investigates LLM self-verification in three formal benchmarks--Game of 24, Graph Coloring and Planning, and shows that accuracy

Two Yochanites drove 15 hours to Austin city limits and saw this.. 🤗

A research note describing our evaluation of the planning capabilities of o1 🍓 is now on arXiv.org arxiv.org/abs/2409.13373 (thanks to Karthik Valmeekam & kstechly). As promised, here is a summary (..although you should read the whole thing..) 🧵 1/

Woo hoo.. Chain of Thoughtlessness paper will be showing up at #NeurIPS2024 🤗 Congrats to Karthik Valmeekam kstechly [details below 👇]