Propensity Score Matching Underestimates Real Treatment Effect, in a Simulated Theoretical Multivariate Model

Abstract

Propensity Score Matching (PSM) is a useful method to reduce the impact of Treatment-Selection Bias in the estimation of causal effects in observational studies. After matching, the PSM significantly reduces the sample under investigation, which may lead to other possible biases (due to overfitting, excess of covariation or a reduced number of observations). In this sense, we wanted to analyze the behavior of this PSM compared with other widely used methods to deal with non-comparable groups, such as the Multivariate Regression Model (MRM). Monte Carlo Simulations are made to construct groups with different effects in order to compare the behavior of PSM and MRM estimating these effects. In addition, the Treatment Selection Bias reduction for the PSM is calculated. With the PSM a reduction in the Treatment Selection Bias is achieved (0.983 [0.982, 0.984]), with a reduction in the Relative Real Treatment Effect Estimation Error (0.216 [0.2, 0.232]), but despite this bias reduction and estimation error reduction, the MRM reduces this estimation error significantly more than the PSM (0.539 [0.522, 0.556], p < 0.001). In addition, the PSM leads to a 30% reduction in the sample. This loss of information derived from the matching process may lead to another not known bias and thus to the inaccuracy of the effect estimation compared with the MRM.

Keywords

• propensity score matching
• multivariate analysis
• general linear model
• Monte Carlo method
• causal effect estimation
• observational study