. Ali Al-Kaswan 🍉 presenting our recent work (w Arie van Deursen பேராசிரியர் Prem Devanbu Toufique Ahmed & @anandsaw) on extending pre-trained models for source code to summarise decompiled binaries, happening now at IEEE SANER ‘23 in Macau. A collaboration between TU Delft and UC Davis. SERG TU Delft

Codex and other LLMs generate a lotta code, for a lotta people. But do they help avoid coding errors? We study this question (with Kevin Jesse, Toufique Ahmed , and Emily Morgan ) in our MSR 2023 paper, using the SStubs4J dataset..we find...

Check out our MSR 2023 paper: Large Language Models and Simple, Stupid Bugs! In this work with Toufique Ahmed , பேராசிரியர் Prem Devanbu , and Emily Morgan we explore Codex and other LLMs code completions in the context of bug prone prompts. While Codex avoids some simple, stupid bugs (SStuBs)

Building reliable hyper-scale cloud services can be challenging. The M365 System Innovation research group advances the understanding of production incidents and uses state-of-the-art AI/ML technologies to help automate cloud operations. Learn more: msft.it/6017gMI7S

I am pleased to announce that our paper, titled 'Better patching using LLM prompting, via Self-Consistency,' co-authored by பேராசிரியர் Prem Devanbu and myself, has been accepted for presentation at ASE 2024 NIER track. Stay tuned for the the pre-print.

Happy to summarize a new ASE-NIER paper, with Toufique Ahmed on using “self-consistency” to boost the defect-patching performance of LLMs for Code. 1/ 🧵 arxiv.org/pdf/2306.00108…

It is really cool!

Retweet appreciated!

Beyond Boundaries: A Conversation with Premkumar Devanbu, Harlan D. Mills Award Award Recipient bit.ly/3T3i2Dk

LLM-generated code saves time; but using this code directly risks injecting errors. So what should devs do? Reviewing it all super-carefully is likely too expensive---so… when to review, and when to just accept? Our ICSE 25 paper is on just this:1/ 5 arxiv.org/pdf/2402.02047

New MSR 2025 paper! SE Research often relies on human subjects to evaluate tool or process innovations. Do they actually help? Questions like “Does this tool generate a good code summary?” “Is this a good, descriptive identifier name?” depend on Human subjective opinion. 1/🧵

Recent SWE agents generate code to resolve issues. While great for productivity, such systems make good tests even more important. To help with that, we present "Otter: Generating Tests from Issues to Validate SWE Patches" & open-source a benchmark, TDD-Bench Verified. 1/5

SWT-Verified ✅ is now released, and Otter establishes a new SOTA 🦦 We have created SWT-Bench Verified, a set of 433 human-verified tasks based on SWE-Bench Verified. "Otter", a new testing agent by IBM Research, establishes a new SOTA here, outperforming @allhands_ai!