Outcomes Affect Evaluations of Decision Quality: Replication and Extensions of Baron and Hershey’s (1988) Outcome Bias Experiment 1

Abstract

Outcome bias is the phenomenon whereby decisions which resulted in successful outcomes were rated more favorably than when the same decisions resulted in failures. We conducted a pre-registered replication and extension of Experiment 1 (original’s: N = 20) from the classic Baron and Hershey (1988) with an online Amazon Mechanical Turk sample using CloudResearch (N = 692), switching from a within-participants design in the original experiment to a between-participants design. We tested outcome bias by measuring participants’ ratings of the quality of decisions in medical scenarios. For the replication (pre-registered) part of the study, we successfully replicated signal and direction of the outcome bias (original: dpaired = 0.21 – 0.53; replication: dindependent = 0.77 [0.62, 0.93] to 1.1 [0.94, 1.26]), and even for participants who stated that outcomes should not be taken into consideration when evaluating decisions (d = 0.64 [0.21, 1.08]). For the extension part of the study, we found differences, dependent on outcome types, in evaluations of the perceived importance of considering the outcome, the perceived responsibility of decision-makers, and the perception that others would act similarly given the choice by outcome type. Materials, data, and code are available on Open Science Framework (OSF): https://osf.io/knjhu/.

Keywords

• outcome bias
• evaluation of decisions
• responsibility
• cognitive biases
• pre-registered replication
• open science