Analysis of the mass survey data reliability (using the example of smoking prevalence in Russia estimates)

Abstract

Survey-based estimates of smoking prevalence in Russia, given by various quite respectable organizations for the same period, range from less than 20% to more than 50%. This scatter cannot be explained by differences in the conceptualization of the indicator (who is considered a smoker) and significantly affects the overall picture of smoking in Russia and the assessment of the success of anti-tobacco policy. This raises questions about the reliability of the relevant survey data. It can be solved in at least two ways: by analyzing and eliminating discrepancy factors or by choosing one of the studies as a reference with a thorough check of the reliability of its data. The article substantiates the choice of the second path and the Russia Longitudinal Monitoring Survey (RLMS) as a reference. To check its reliability, three criteria were used: analysis of critical publications about research errors, methodological triangulation (comparison with data from other methods) and analysis of correlation with the consequences of smoking. All three criteria showed a high level of reliability of the RLMS data. The hypothesis about the research error put forward in critical publications does not find empirical confirmation. Comparison with other supposedly reliable data (both survey and non-survey) shows discrepancies below the margin of statistical error. The correlation of RLMS data on smoking with mortality from diseases of the circulatory system turns out to be extremely high. Therefore, it is advisable to consider the RLMS as a reference source of data on the prevalence of smoking in Russia, and indicators from other surveys that differ significantly from it as potentially questionable in their methodological decisions.

Keywords

• smoking prevalence
• social surveys
• reliability of survey data
• methodological triangulation
• rlms