This post is a summary of our ICSE-SEIP'23 paper,

Exploring 50 user interfaces for AI code suggestions

submited by

Style Pass

2024-05-07 21:00:16

This post is a summary of our ICSE-SEIP'23 paper, "Towards More Effective AI-Assisted Programming: A Systematic Design Exploration to Improve Visual Studio IntelliCode's User Experience". See the paper for more details. Thanks to Priyan Vaithilingam for leading this project!

What is the optimal way to present these code suggestions to users? Showing the suggestion as inline gray text is common. But are there better ways? What if it is a multi-line change that modifies existing code?

We iteratively explored 50 or so designs for inline code change interfaces in Visual Studio. We filtered our designs down to 19 which we then tested in a series of 7 lab studies involving 61 developers.

The lab studies involved small coding tasks like adding a new property or refactoring. The developers were not informed of the AI code suggestion tool. We used Visual Studio's lightbulb as a baseline interface, where an icon by the line of code pops up and can be clicked to view suggestions.

Finally, we conducted a large-scale deployment of the two most promising designs in Visual Studio 2022. One design was released in a public release and the other in an opt-in preview release.