From Whence Commeth Data Misreporting? A Survey of Benford’s Law and Digit Analysis in the Time of the COVID-19 Pandemic

Abstract

We survey the literature on the use of Benford’s distribution digit analysis applied to COVID-19 case data reporting. We combine a bibliometric analysis of 32 articles with a survey of their content and findings. In spite of combined efforts from teams of researchers across multiple countries and universities, using large data samples from a multitude of sources, there is no emerging consensus on data misreporting. We believe we are nevertheless able to discern a faint pattern in the segregation of findings. The evidence suggests that studies using very large, aggregate samples and a methodology based on hypothesis testing are marginally more likely to identify significant deviations from Benford’s distribution and to attribute this deviation to data tampering. Our results are far from conclusive and should be taken with a very healthy dose of skepticism. Academics and policymakers alike should remain mindful that the misreporting controversy is still far from being settled.

Keywords

• COVID-19 data
• Benford’s Law
• statistical tests
• MAD
• data conformity
• research methodology