Learning to Rank Reviewers for Pull Requests

Abstract

In pull-based software development, anyone who wants to contribute to a project can request integration of the code changes to the public repository by sending a pull request to the development team. Upon receiving a pull request, a team member will review the changes and decide on merging it or not in the repository. Finding the appropriate reviewer for a pull request is a crucial step. To support reviewer recommendation, this paper introduces an adaptive ranking model to rank all the reviewer candidates for a pull request. The ranking model leverages 14 features to measure the relationships between a pull request and the reviewer candidates. The weight parameters of the ranking model are trained automatically based on previously resolved pull requests by using a learning-to-rank technique. The experimental evaluations on 12 open-source projects show that the proposed approach outperforms the baseline and a state-of-the-art approach. It can recommend the suitable reviewers within top-1 recommendation for over 80% of the pull requests in the opencv and jekyll projects. The feature selection experiments show that the most important feature is the feature that counts the number of previous pull requests sent by the requester and reviewed by a developer. The feature that measures the file path similarity between files changed in the pull request and files previously modified by a developer is another important feature.

Keywords

• Learning to rank
• pull request
• reviewer recommendation