Evaluating Low-Cost in Internal Crowdsourcing for Software Engineering: The Case of Feature Location in an Industrial Environment

Abstract

Internal crowdsourcing in software engineering is a mechanism for recruiting engineers to carry out more efficiently software engineering tasks. However, engineers are busy resources and time is a valuable asset in industry, which hinders internal crowdsourcing in software engineering from becoming a widespread practice. In this work, we propose a low-cost variant of internal crowdsourcing for locating features in models, which limits the time that engineers can spend for providing knowledge. Our approach uses the knowledge provided by the internal crowd to automatically reformulate an initial feature description. The result is taken as input to automatically locate the relevant model fragment using Latent Semantic Indexing. We evaluate our approach using four query reformulation techniques in a real-world case study from our industrial partner. We compare the results of our approach in terms of recall, precision and F-measure with a baseline by means of statistical methods to show that the impact of the results of our approach is significant. Despite the limitation of time, the results show that low-cost in internal crowdsourcing improves significantly the results in an industrial context where engineers' availability is scarce.

Keywords

• Crowdsourced software engineering
• crowdsourcing
• collaborative information retrieval
• query reformulation
• feature location