🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

I'll be presenting my #NAACL2024 work:✨ADaPT✨ in-person 🇲🇽 tomorrow (June 19) at 11 AM in poster session 7! ADaPT enables LLMs to "adapt" to task complexity & execution failures by decomposing recursively w/ Alexander Koller M Hartmann, P Clark Ashish Sabharwal Mohit Bansal tusharkhot

Can LLMs help accelerate the discovery of data-driven scientific hypotheses? 🧬📊 We benchmark this in DiscoveryBench: 264 discovery tasks from 6 scientific domains, from humanities to biology: arxiv.org/pdf/2407.01725… Ai2 Aristo Team at AI2 Harshit Surana UMass Amherst

To y'all attending #icml24, gearing up to argue what general-purpose generative models/agents can and cannot do for automated scientific discovery. Would love to hear your thoughts and counter-arguments on Wed (7/24) at 1:30, Hall C 4-9 #117. + discuss what's cooking Ai2

Symbolic planners do explicit search for most steps. However, LLMs can solve easy sub-goals directly (Sys-1) and verbalize search for harder ones (Sys-2). In✨System-1.x✨, we train LLMs to efficiently balance Sys-1 & 2. Users can simply control hybridization via the dial x! 🎛️

AppWorld A Controllable World of Apps and People for Benchmarking Interactive Coding Agents Autonomous agents that address day-to-day digital tasks (e.g., ordering groceries for a household), must not only operate multiple apps (e.g., notes, messaging, shopping app) via

🔥 Autonomous AI Assistants (e.g., #googleio2024, #WWDC24) and coding agents (e.g., #Devin, #SWEAgent) have garnered a lot of attention recently. We can envision coding agents autonomously completing complex day-to-day tasks across apps using APIs on our behalf. But how can we

AppWorld won the (one of the) best resource paper award(s) at #ACL2024 Outstanding resource and great work by Harsh Trivedi Niranjan at Stony Brook tusharkhot Ashish Sabharwal Aristo Team at AI2 and collaborators 🧵👇

🏆 ACL Best Theme Paper Award: - OLMo: Accelerating the Science of Language Models by Groeneveld et al. #NLProc #ACL2024NLP

🏆 AppWorld won a #ACL2024NLP Best Resource Paper Award. 🥳 Congrats team! I'm so happy for Harsh Trivedi. The time & care he put in is inspiring. #proudadvisor 🚨He is on the job market.🚨 Hire him! 🌐Check out appworld.dev Stony Brook University Dept. of Computer Science @AI_SBU #NLProc Ai2

📢 New Benchmark: SUPER for Setting UP and Executing tasks from Research repositories Reproducibility is crucial in science. We introduce SUPER to evaluate LLMs' capabilities in autonomously running experiments from research repositories. ⬇️ arxiv.org/pdf/2409.07440

Meet Molmo: a family of open, state-of-the-art multimodal AI models. Our best model outperforms proprietary systems, using 1000x less data. Molmo doesn't just understand multimodal data—it acts on it, enabling rich interactions in both the physical and virtual worlds. Try it

📢 I am giving a talk on 🌎 AppWorld & its future works at 13+ universities (USC, UCI, Stanford, Berkeley, Princeton, JHU, ..) and companies (Ai2, Google, Apple, Semantic Machines, ..) in the next 1-2 months 📅 Schedule+Details: appworld.dev/talks x.com/harsh3vedi/sta…

Can language models perform end-to-end scientific discovery? In our NeurIPS Spotlight paper, we show: very rarely. Our best model found <20% of discoveries, our best PhDs found nearly all. Paper: arxiv.org/pdf/2406.06769 Code/Web: allenai.github.io/discoveryworld Ai2 Microsoft Research

For this week’s NLP Seminar, we are thrilled to host Harsh Trivedi to talk about AppWorld: Reliable Evaluation of Interactive Agents in a World of Apps and People! When: 10/10 Thurs 11am PT Non-Stanford affiliates registration form: forms.gle/UjWyX6dn7mQafj… (closed at 9am PT on

🚨Looking to self-align models on complex problem-solving tasks without gold answers or labels? Checkout my internship work on ✨Self-consistency Preference Optimization (ScPO)✨, where we use the self-consistency concept to help train models by iteratively training consistent

Introducing Deep Research, your personal agentic AI research assistant. Rolling out starting today in Gemini Advanced. With Deep Research, you can create in-depth research reports on complex topics, complete with source links, giving you hours of research at your fingertips in

Working on Scientific discovery? Submit your papers to the AI & Scientific Discovery workshop. Due: Jan 30!

AI & Scientific Discovery Workshop deadline extended, with a few more days to submit! Submit your archival & non-archival papers broadly in the AI & Scientific Discovery space, for a great opportunity to attend a workshop with like-minded folks. ai-and-scientific-discovery.github.io

If you're interested in LLMs like o1 and R1 for complex reasoning, check out this paper — we show that logical reasoning tasks are ideal for evaluating and understanding their scaling limits. 🦓 ZebraLogic-Bench is a dataset of 1K constraint satisfaction problems (CSPs)

We’re also rolling out a version of 2.0 Flash Thinking that can interact with apps like YouTube, Google Search and @GoogleMaps. These connected apps already make the Gemini app a uniquely helpful AI-powered assistant, and we’re exploring how new reasoning capabilities can