The Potentials of Cochrane Reviewers' Comments and Citation Contexts in the Recognition of Randomized Controlled Trials' Texts and their Main Sections

Abstract

Introduction: Meta-textuality can provide effective medium for facilitating information retrieval and machine learning. This study explored the strengths of two types of meta-texts (i.e., reviewers' comments and citation contexts) in correct classification and recognition of their related texts and main sections at abstract level. Methods: In this descriptive study with quantitative content analysis method, 846 randomized controlled trials were assessed; and their reviewers' comments and citation contexts were extracted from Cochrane reviews and Colil databases. Thirty seed documents were randomly selected as queries, and their abstract similarities to the test collection and the main sections (IMRaD: introduction, method, results, discussion) were calculated. Receiver operating characteristic (ROC) was used to analyze the performance of Cochrane reviewers' comments and citation contexts individually and in combination. Results: The citation contexts’ area under the curve (0.807) was higher than the Cochrane comments' (0.638), and reached its highest for their combination (0.936). The former had the highest performance in correct classification of the introduction section (0.661), and the latter in correct recognition of the methodology section (0.606). Conclusion: Cochrane reviewers’ comments and the citation contexts had the potential of correct classification of the related texts. The former did well in identifying the methodology section, while the latter in identifying the introduction section. Combining the two systems can boost their power in identifying the discussion section. The results can have implications for natural language processing, machine learning systems, text categorization, retrieval, and summarization.

Keywords

• text similarity
• cochrane reviewers' comments
• citation contexts
• natural language processing