Necessary condition analysis has either low specificity or low sensitivity: Results from simulations and empirical analyses of grit, depression, and anxiety

Abstract

Objectives: Initially the stated goal of Necessary Condition Analysis (NCA) was to help identify conditions that are necessary but not sufficient for some outcome. However, later the developers of the test asserted that the test is for identifying if the association between two variables is characterized by some indeterminate type of non-randomness. The objective of the present study was to assess the ability of NCA to achieve its originally as well as its more newly stated objective. Furthermore, the performance of NCA was compared with the performance of ordinary linear regression analysis. Methods: Data simulating various deviations from randomness as well as empirical data on grit, depression, and anxiety in the 1997 National Longitudinal Survey of Youth (NLSY97) were analyzed with NCA and with linear regression. Results: For its initially stated objective, NCA displayed low specificity. For its more newly stated objective, NCA exhibited low sensitivity. Ordinary linear regression analysis was better than NCA at identifying non-random associations, especially negative associations. Conclusions: There does not appear to exist any convincing reasons to use the significance test in NCA instead of ordinary linear regression analysis. There appears to be confusion about how results from NCA should be interpreted, maybe even among the developers of the test.

Keywords

• Linear regression analysis
• Necessary condition analysis (NCA)
• Sensitivity
• Specificity
• Statistical performance