Hey, so compiler fuzzers find corner cases by generating highly unnatural code, right? Maybe NOT. We found that CSmith's programs are *more* predictable than human-written ones, as per language models trained on GitHub code. Check out our @msconf preprint: rohan.padhye.org/files/natfuzz-…

Congrats to my undergraduate student Bella Laybourn for receiving the second prize in the ACM student research competition at #ICSE2022 for Mu2: Greybox Fuzzing with Mutation Analysis. Bella started this project as a sophomore student at CMU. Great job!

chatgpdf with Vasu Vikram 🙃 anyone wanna guess what's going on?

JQF 2.0 released! The Java fuzzing framework now incorporates several speed and quality improvements from Jonathan Bell & co. as well as support for implementing differential fuzzing. I'm excited to see lots of recent work leveraging #JQF's extensibility. github.com/rohanpadhye/JQF

I'm very excited to present our work on Mu2 at ISSTA Conference this July! Would love to hear everyone's thoughts on how we could use fuzzing to create strong regression tests. Adding mutation analysis in the loop may be the first step!

Do you like quickcheck/hypothesis/fuzzing but can't bother to hand-write oracles/properties that validate the correctness of your APIs? Check out proptest.ai! Simply copy+paste Python API docs from a website and let an LLM have a go at synthesizing a property test.

Very excited to announce that our work on Mu2 received a Distinguished Paper Award at ISSTA Conference! 🎉 Come check out my talk tomorrow at 1:30pm PDT during the Fuzzing session : )

If you found this interesting/want to dig deeper, Vasu Vikram, Michael Zhou and I built a lecture+project for a SWE class at CMU School of Computer Science. [Lecture] A software engineer's guide to LLM's-cmu-313.github.io/_old/F23/asset… [Project] Answer our syllabus q's with an LLM-colab.research.google.com/drive/18ppvk_X…

📢 FUZZING NEWS! For many years Caroline Lemieux and I got asked whether JQF/Zest mutations always preserve input structure. THEY DON'T! There's sometimes a disruptive "havoc" effect. BeDivFuzz (Hoang Lam Nguyen) and Zeugma (Jonathan Bell) do a much better job avoiding it. But WAIT....

... does it really matter in practice? Perhaps the havoc effect is benign, or maybe even a good thing?! Our students Ao Li Vasu Vikram Madonna Huang studied this all in great depth! Read more in our upcoming ACM TOSEM paper on the "Havoc Paradox". 📄 dl.acm.org/doi/pdf/10.114…