Addressing recall bias in (post-)conflict data collection and analysis: lessons from a large-scale health survey in Colombia

Abstract

Abstract Background Much applied research on the consequences of conflicts for health suffers from data limitations, particularly the absence of longitudinal data spanning pre-, during- and post-conflict periods for affected individuals. Such limitations often hinder reliable measurement of the causal effects of conflict and their pathways, hampering also the design of effective post-conflict health policies. Researchers have sought to overcome these data limitations by conducting ex-post surveys, asking participants to recall their health and living standards before (or during) conflict. These questions may introduce important analytical biases due to recall error and misreporting. Methods We investigate how to implement ex-post health surveys that collect recall data, for conflict-affected populations, which is reliable for empirical analysis via standard quantitative methods. We propose two complementary strategies based on methods developed in the psychology and psychometric literatures—the Flashbulb and test-retest approaches—to identify and address recall bias in ex-post health survey data. We apply these strategies to the case study of a large-scale health survey which we implemented in Colombia in the post-peace agreement period, but that included recall questions referring to the conflict period. Results We demonstrate how adapted versions of the Flashbulb and test-retest strategies can be used to test for recall bias in (post-)conflict survey responses. We also show how these test strategies can be incorporated into post-conflict health surveys in their design phase, accompanied by further ex-ante mitigation strategies for recall bias, to increase the reliability of survey data analysis—including by identifying the survey modules, and sub-populations, for which empirical analysis is likely to yield more reliable causal inference about the health consequences of conflict. Conclusions Our study makes a novel contribution to the field of applied health research in humanitarian settings, by providing practical methodological guidance for the implementation of data collection efforts in humanitarian contexts where recall information, collected from primary surveys, is required to allow assessments of changes in health and wellbeing. Key lessons include the importance of embedding appropriate strategies to test and address recall bias into the design of any relevant data collection tools in post-conflict or humanitarian contexts.

Keywords

• Conflict
• Health
• Household surveys
• Recall bias
• Causal effects
• Survey data