Crowd Controls Crowd: Quality Improvement of Polygon Integration in Paid Crowdsourcing

Abstract

In this study, we introduce a crowd-driven data enhancement strategy for the integration of polygons in paid crowdsourcing. First, we capture redundant polygons with a web-based tool using one set of crowdworkers. Then, we present the acquired polygons to other crowdworkers in a polygon editing tool, with instructions to validate and improve those acquisitions. This procedure is repeated by showing a third set of crowdworkers the already edited polygon geometries. Furthermore, a polygon integration procedure is performed for every step, i.e., the unedited polygons, those edited once and those edited twice, allowing for a comparative qualitative analysis. This analysis is conducted with a focus on both quality and cost control, aiming for small sample sizes in order to optimize costs. Additionally, we conduct further investigations to assess the effectiveness of our approach in different use cases, and explore potential adaptions for further enhancement.