Are paid tools worth the cost? A prospective cross-over study to find the right tool for plagiarism detection

Abhishek Anil; Aswini Saravanan; Surjit Singh; Muhammad Aaqib Shamim; Krishna Tiwari; Hina Lal; Shanmugapriya Seshatri; Simi Bridjit Gomaz; Thoyyib P. Karat; Pradeep Dwivedi; Shoban Babu Varthya; Rimple Jeet Kaur; Prakasini Satapathy; Bijaya Kumar Padhi; Shilpa Gaidhane; Manoj Patil; Mahalaqua Nazli Khatib; Joshuan J. Barboza; Ranjit Sah

Heliyon (Sep 2023)

Are paid tools worth the cost? A prospective cross-over study to find the right tool for plagiarism detection

Abhishek Anil,
Aswini Saravanan,
Surjit Singh,
Muhammad Aaqib Shamim,
Krishna Tiwari,
Hina Lal,
Shanmugapriya Seshatri,
Simi Bridjit Gomaz,
Thoyyib P. Karat,
Pradeep Dwivedi,
Shoban Babu Varthya,
Rimple Jeet Kaur,
Prakasini Satapathy,
Bijaya Kumar Padhi,
Shilpa Gaidhane,
Manoj Patil,
Mahalaqua Nazli Khatib,
Joshuan J. Barboza,
Ranjit Sah

Affiliations

Abhishek Anil: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Aswini Saravanan: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Surjit Singh: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Muhammad Aaqib Shamim: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Krishna Tiwari: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Hina Lal: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Shanmugapriya Seshatri: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Simi Bridjit Gomaz: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Thoyyib P. Karat: Department of Dermatology, Venereology and Leprosy, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Pradeep Dwivedi: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Shoban Babu Varthya: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Rimple Jeet Kaur: Department of Pharmacology, All India Institute of Medical Sciences, Jodhpur, Rajasthan-342005, India
Prakasini Satapathy: Global Center for Evidence Synthesis, Chandigarh-160036, India
Bijaya Kumar Padhi: Department of Community Medicine and School of Public Health, Postgraduate Institute of Medical Education and Research, Chandigarh - 160012, India
Shilpa Gaidhane: One Health Centre (COHERD), Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education, Wardha - 442001, India
Manoj Patil: Division of Evidence Synthesis, School of Epidemiology and Public Health and Research, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education, Wardha - 442001, India
Mahalaqua Nazli Khatib: Division of Evidence Synthesis, Global Consortium of Public Health and Research, Datta Meghe Institute of Higher Education and Research, Wardha - 442001, India; Corresponding author. Division of Evidence Synthesis, Global Consortium of Public Health and Research, Datta Meghe Institute of Higher Education, Wardha - 442001, India.
Joshuan J. Barboza: Escuela de Medicina, Universidad Cesar Vallejo, Trujillo, 13007, Peru; Corresponding author.
Ranjit Sah: Tribhuvan University Teaching Hospital, Kathmandu - 46000, Nepal; Department of Clinical Microbiology, DY Patil Medical College, Hospital and Research Centre, DY Patil Vidyapeeth, Pune - 411000, Maharashtra, India; Department of Public Health Dentistry, Dr. D.Y. Patil Dental College and Hospital, Dr. D.Y. Patil Vidyapeeth, Pune - 411018, Maharashtra, India; Corresponding author. Infectious Diseases Fellowship, Clinical Research (Harvard Medical School), Global Clinical Scholars Research Training (Harvard Medical School), Systemic Review and Meta-analysis (London School of Hygiene and Tropical Medicine), Editorial Board Member: TMAID, ID Cases, BMC Infectious Diseases, Tribhuvan University Teaching Hospital, Kathmandu - 46000, Nepal.

Journal volume & issue: Vol. 9, no. 9
p. e19194

Abstract

Read online

Background: The increasing pressure to publish research has led to a rise in plagiarism incidents, creating a need for effective plagiarism detection software. The importance of this study lies in the high cost variation amongst the available options for plagiarism detection. By uncovering the advantages of these low-cost or free alternatives, researchers could access the appropriate tools for plagiarism detection. This is the first study to compare four plagiarism detection tools and assess factors impacting their effectiveness in identifying plagiarism in AI-generated articles. Methodology: A prospective cross-over study was conducted with the primary objective to compare Overall Similarity Index(OSI) of four plagiarism detection software(iThenticate, Grammarly, Small SEO Tools, and DupliChecker) on AI-generated articles. ChatGPT was used to generate 100 articles, ten from each of ten general domains affecting various aspects of life. These were run through four software, recording the OSI. Flesch Reading Ease Score(FRES), Gunning Fog Index(GFI), and Flesch-Kincaid Grade Level(FKGL) were used to assess how factors, such as article length and language complexity, impact plagiarism detection. Results: The study found significant variation in OSI(p < 0.001) among the four software, with Grammarly having the highest mean rank(3.56) and Small SEO Tools having the lowest(1.67). Pairwise analyses revealed significant differences(p < 0.001) between all pairs except for Small SEO Tools-DupliChecker. Number of words showed a significant correlation with OSI for iThenticate(p < 0.05) but not for the other three. FRES had a positive correlation, and GFI had a negative correlation with OSI by DupliChecker. FKGL negatively correlated with OSI by Small SEO Tools and DupliChecker. Conclusion: Grammarly is unexpectedly most effective in detecting plagiarism in AI-generated articles compared to the other tools. This could be due to different softwares using diverse data sources. This highlights the potential for lower-cost plagiarism detection tools to be utilized by researchers.

Published in Heliyon

ISSN: 2405-8440 (Online)
Publisher: Elsevier
Country of publisher: United Kingdom
LCC subjects: Science: Science (General); Social Sciences: Social sciences (General)
Website: https://www.cell.com/heliyon/home

About the journal

Abstract

Keywords