Hiroshi Ishii　石井 裕 Elizabeth Churchill R. Lisa Huang John Chen Litao Yan Standing room only for Hussein Mozannar et al. Microsoft Research with another Copilot paper! They build a 'CUPS' framework for labeling how AI-assisted coding time is spent. There is clearly a ton of room for making coding even more productive with only current models! #CHI2024

ece8bb Michael Bernstein Stephanie Bell Su Lin Blodgett Do you wish we had evals for LLM performance with real humans instead of just superficial, ephemeral benchmarks? Hussein Mozannar presents RealHumanEval, a platform with a LeetCode-style IDE to evaluate in vivo. They show benchmark %s can be way off from real performance. #CHI2024

Presenting at #CHI2024 :
1) RealHumanEval - our platform for human evaluation of LLMs for code, at TREW workshop arxiv.org/abs/2404.02806
2) Reading Between The Lines: Modeling AI-Assisted Programming (Honorable Mention Award) - Tuesday 2:45pm in 324 arxiv.org/abs/2210.14306

Feeling some FOMO that I won't be attending #chi2024 this year. For those who are, be sure to say👋 to the fabulous Gagan Bansal , Adam Fourney (hci.social/@adam), and the newest member of the #HAXTeam at #MSR AI Frontiers, Hussein Mozannar! Don't forget to ask them about opportunities on our team👇

Terrrrrific work, Dr. Hussein Mozannar! Hussein Mozannar It was an honor to serve on your committee, and I can't wait to see what you do next!

So happy to defend my PhD thesis and couldn't have done it without a 1 of 1 advisor David Sontag and an incredible committee Eric Horvitz Arvind Satyanarayan Elena Glassman

Congratulations to Dr. Hussein Mozannar. Hussein Mozannar Strong dissertation research exploring multiple dimensions of human-AI collaboration. David Sontag Elena Glassman Arvind Satyanarayan Massachusetts Institute of Technology (MIT)

I will be defending my PhD thesis, Training Human-AI Teams, on April 25th at MIT and on Zoom! Please DM me or email for a link! -- (image generated after an hour of prompting DALL·E)

Sneak peak!

Just recorded a demo
youtu.be/3sKIoVp8QRw

Some alpha users found a ton of bugs, so beta release is a bit delayed till next week at least.

You can still sign up @ dataline.app!

It's already open source but not ready for full public release yet 😁

Elena Glassman Hussein Mozannar Minsuk Chang This is such a great thread!! Two classic readings to understand what “theory” means for HCI are: link.springer.com/book/10.1007/9… and dl.acm.org/doi/10.1145/34…

Beyond the quality of our decisions, how will AI assistance affect us -- our growth and improvement, enjoyment, collaboration, or our agency in the workplace? The current design of AI assistance does not consider human-centric objectives; we need methods to account for them.

Super cool from our lab for selectively decoding between multiple LLMs at each token!

Our paper on adapting Copilot from human feedback is at #AAAI24 !
One of the secret sauces to making GitHub Copilot work is knowing when to show suggestions, our paper presents a motivated approach (CDHF) to achieve this using feedback data
arxiv.org/pdf/2306.04930…

We show in our work that LLM augmentations can improve clinical note readability! I was mostly involved with quantitative evaluation, but there are fascinating insights about LLM errors and qualitative insights in interviews with breast cancer survivors arxiv.org/abs/2401.09637

👩‍⚕️💻 Our physician-machine partnerships paper is out in Nature Medicine today!

Experiment w/ 1000+ dermatologists & PCPs shows how AI assistance can boost diagnostic accuracy but reveals limits to addressing physician bias w/ fair classifiers

nature.com/articles/s4159…

Thread 👇

A new onboarding technique can help workers collaborate more effectively with AI assistants. The process finds situations where the human trusts the AI either too much or too little, then develops rules about them and creates training exercises. mitsha.re/cKP350Qqw2j