Nongye tushu qingbao xuebao (Jul 2023)
Key Parameter Optimization Design of Self-organizing Peer Review in National Preprint Publishing Platform Based on Response Surface Analysis
Abstract
[Purpose/Significance] The National Preprint Publishing Platform of China had been put into operation by the end of 2022. In order to ensure the quality of papers posted on the Platform and encourage a large number of researchers to submit articles to the Platform, it is necessary to introduce a paper quality assurance mechanism into the Platform. There are deficiencies in existing quality assurance mechanism for a preprint publishing platform. Self-organizing peer review has gradually attracted attention in recent years. In order to achieve the performance expected by the management agency of the National Preprint Publishing Platform, it is necessary to study the optimization design of key parameters of self-organizing peer review. [Method/Process] Design-expert software has been used for Box Behnken experimental design. The number of papers submitted by authors within a year, the distribution of review time, and the parameters of article review time are randomly set. The total number of papers is calculated according to the parameters of Lotka's law, and is not included in the Box-Behnken experimental sampling. We selected four key parameters of rest time, rejection rate, number of researchers, and review qualification ratio as independent variables, took three levels for each variable, and used -1, 0, +1 for coding, representing three levels of low, medium, and high, respectively. The three dependent variables for self-organizing peer review performance evaluation are the completion rate of paper review (CR), the balance of reviewer task allocation (TBD), and the average review time of the paper (A_r_time). A Box-Behnken experiment table was designed separately for each Lotka's law parameter, three repeated experiments were conducted on each key parameter combination in the table, and the summary() function and mean() function in R language were used to calculate the average performance indicators of each key parameter combination. Then, the response surface analysis was carried out on the Box-Behnken sampling data, and the quantitative relationship expression between the dependent variable of performance evaluation and the independent variable of key parameters was obtained. [Results/Conclusions] The ratio of review qualifications and reviewer rest time have greater impact on the performance of self-organized peer review, especially the ratio of review qualifications is much more important than other three key parameters. By combining different values of key parameter independent variables, the National Preprint Publishing Platform can achieve the expected self-organized peer review performance, ensuring the average quality of papers published by the Platform. In addition, because of our limited knowledge of computational resources, there is still room for improvement in simulation accuracy. If equipped with more powerful computational resources, more precise simulation calculation can be carried out based on the methods and parameters proposed in this paper.
Keywords
