Do we focus on process over outcome? Review of published studies in two prominent saudi journals

Abstract

It is well-known that published science is judged by the judicious and careful use of methods employed when acquiring knowledge and understanding. However, since the beginning of the millennia, concerns over the validity and reliability of scientific research have grown substantially. These concerns were mainly related to statistical methods and interpretation of statistical inferences. This study attempts to investigate the statistical practices of published studies in two prominent Saudi journals. The purpose is to analyze the extension of the problem into a different geographical area. In addition, it attempts to look at the established editorial procedures utilized in the educational and linguistics sciences in Saudi Arabia. 114 published articles between 2017 and 2020 were analyzed by a 13-item checklist. The findings revealed an over-reliance on p-value and almost 67% of the published studies interpreted p-value as a proven population effect. In addition, researchers and editors relied heavily on a significant p-value as a merit for publication. Moreover, there was a low reporting rate in incorporating other statistical requirements such as CI and effect size. The implication of the findings highlights the importance of a more “verifying: how you found it” approach rather than over-reliance that significant p-value is publishable “what you found approach”.

Keywords

• p-value
• statistical practices
• replication crisis
• Saudi journals
• journal editorial requirements
• Psychological Methods & Statistics