Satisficing Response Behavior Across Time: Assessing Negative Panel Conditioning Using an Experimental Design with Six Repetitions

Abstract

Satisficing response behavior can be a threat to the quality of survey responses. Past research has provided broad empirical evidence on the existence of satisficing and its consequences on data quality, however, relatively little is known about the extent of satisficing over the course of a panel study and its impact on response quality in later waves. Drawing on panel conditioning research, we use question design experiments to investigate whether learning effects across waves of a panel study cause changes in the extent of satisficing and if so, whether general survey experience (process learning) or familiarity with specific question content (content learning) accounts for those changes. We use data from a longitudinal survey experiment comprising six panel waves administered within a German non-probability online access panel. To investigate the underlying mechanism of possible learning effects, the experimental study randomly assigned respondents to different frequencies of receiving identical question content over the six panel waves. Our results show the existence of satisficing in every panel wave, which is in its magnitude similar to the extent of satisficing in the probability-based GESIS Panel that we use as a benchmark study. However, we did not find changes in the extent of satisficing across panel waves, nor did we find moderation effects of the interval between the waves, respondents’ cognitive ability, or motivation. Additional validity analyses showed that satisficing does not only affect the distribution of individual estimates by 15 percent or more but also can have an effect on associations between variables.

Keywords

• Panel Conditioning
• Satisficing
• learning effects
• non-probability panel
• response quality
• form-resistant correlation hypothesis