A robust effect size measure for MANOVA with non-normal and non-homogenous data

Abstract

A common research question in psychology entails examining whether significant group differences (e.g. male and female) can be found in a list of numeric variables that measure the same underlying construct (e.g. intelligence). Researchers often use a multivariate analysis of variance (MANOVA), which is based on conventional null-hypothesis significance testing (NHST). Recently, a number of quantitative researchers have suggested reporting an effect size measure (ES) in this research scenario because of the perceived shortcomings of NHST. Thus, a number of MANOVA ESs have been proposed (e.g. generalized eta squared η Λ 2 , generalized omega squared ω Λ 2 ), but they rely on two key assumptions—multivariate normality and homogeneity of covariance matrices—which are frequently violated in psychological research. To solve this problem we propose a non-parametric (or assumptions-free) ES ( A w ) for MANOVA. The new ES is developed on the basis of the non-parametric A in ANOVA. To test A w we conducted a Monte-Carlo simulation. The results showed that A w was accurate (robust) across different manipulated conditions—including non-normal distributions, unequal covariance matrices between groups, total sample sizes, sample size ratios, true ES values, and numbers of dependent variables—thereby providing empirical evidence supporting the use of A w , particularly when key assumptions are violated. Implications of the proposed A w for psychological research and other disciplines are also discussed.