Improve Reputation Evaluation of Crowdsourcing Participants Using Multidimensional Index and Machine Learning Techniques

Abstract

Building a scientific and reasonable reputation evaluation mechanism for crowdsourcing participants is an effective way to solve the problem of transaction fraud, to establish the trust of traders and ensure the quality of task completion. Under the big data environment, machine learning methods have been applied in the domain of e-commerce of physical goods to improve the traditional reputation evaluation methods, and achieved good results. However, few studies have applied machine learning methods to crowdsourcing, a form of service e-commerce, to evaluate the reputation of participants. This paper proposes a reputation evaluation model (i.e. LDA-RF) for crowdsourcing participants of Random Forest based on Linear Discriminant Analysis. The model consists of five steps: firstly, building a multidimensional reputation evaluation index system for crowdsourcing participants, collecting real data sets, and preprocessing data; secondly, data dimensionality reduction methods, including Linear Discriminant Analysis, Principal Component Analysis, Mean Impact Value method and ReliefF feature selection method, are used to eliminate redundant variables; thirdly, data normalization; fourthly, with selected feature subset, five machine learning techniques, Random Forest, Decision Tree, Back propagation Neural Network, Radial Basis Function Neural Network and Support Vector Machine are used to train the model; Fifthly, the validity of the model is tested by four evaluation measures: 10 fold cross validation, confusion matrix, Kruskal-wallis test and dispersion degree. The results show that the LDA-RF model on accuracy, F1-measure, generalization ability and robustness are better than those of other models, and it has better performance and effectiveness. This study represents a new contribution to establish reputation evaluation of crowdsourcing participants under big data environment.

Keywords

• Crowdsourcing participants
• reputation evaluation
• machine learning
• random forest
• data dimension reduction