The cluster depth tests: Toward point-wise strong control of the family-wise error rate in massively univariate tests with application to M/EEG

Abstract

The cluster mass test has been widely used for massively univariate tests in M/EEG, fMRI and, recently, pupillometry analysis. It is a powerful method for detecting effects while controlling weakly the family-wise error rate (FWER), although its correct interpretation can only be performed at the cluster level without any point-wise conclusion. It implies that the discoveries of a cluster mass test cannot be precisely localized in time or in space. We propose a new multiple comparisons procedure, the cluster depth tests, that both controls the FWER while allowing an interpretation at the time point level. We show the conditions for a strong control of the FWER, and a simulation study shows that the cluster depth tests achieve large power and guarantee the FWER even in the presence of physiologically plausible effects. By having an interpretation at the time point/voxel level, the cluster depth tests make it possible to take full advantage of the high temporal resolution of EEG recording and give a precise timing of the start and end of the significant effects.

Keywords

• Multiple comparisons procedure
• Cluster mass test
• Permutation test
• Neurosciences
• TFCE