An Eye-Tracking Study on the Use of Split/Unified Code Change Views for Bug Detection

Abstract

Software developers often use comparison tools to identify bugs in source code by comparing two versions of the code. These tools display the code in either a split view or unified manner, yet there is limited understanding of how developers interact with them when analyzing changes in the source code. Consequently, the advantages and disadvantages of each approach are still unknown. We explored the usefulness, limitations, and potential for improvement of split and unified views of GitHub. To do this, we conducted a user study with 12 participants who were tasked with analyzing source code in two commits to detect bugs. We used a within-subjects approach, and the participants were monitored with an eye-tracking device while using the split and unified views. The results of our study showed that participants experienced less visual strain when using the unified view. This implies that a lower effort can facilitate a more thorough analysis of the code, which can often result in the discovery of more bugs. Additionally, we noticed that participants mainly focused on conditionals, class/instance variables, and code changes. Nevertheless, we did not find any statistically significant differences among the variables examined when using split or unified views. The findings of this study are pertinent to both practitioners and researchers. Practitioners can use the results to make informed decisions when selecting a view, while researchers can use them to identify potential areas for improvement in code comparison tools.

Keywords

• Bug detection
• code review
• eye-tracking
• user study