In a recent study, Eklund et al. sparked an ongoing international debate when it highlighted systemic failures in cluster-based analysis of functional magnetic resonance imaging (fMRI). The fMRI method has been used for decades to investigate correlations between brain region inactivation and task performance. Active regions in the brain are assigned by two methods: voxel-wise and cluster-wise inferences. Voxel-wise inference assigns activity to brain regions based on association of specific voxels. Meanwhile, cluster-wise inference assigns activity based on correlation between specific clusters of voxels usually associated by size. The occurrence of false-positives is controlled in the most commonly used fMRI software packages (SPM, FSL and AFNI) by a function known as the Family-wise error (FWE). The Eklund et al. study examined the reliability of the five FWE analysis tools offered by the main software packages. The results showed that for the FWE in cluster-wise inference, parametric studies gave extremely high false-positives but were within range for the voxel-wise inference. To analyze the data using a nonparametric test, Eklund et al. utilized a permutation test which gave results for the FWE within the boundaries for both cluster-wise and voxel-wise inferences.
An independent post examined the assumptions behind the comparison of the five different FWE tools based on the differences between voxel-wise and cluster-wise thresholds. In short, voxel-wise thresholding relies on making a decision about ‘active’ brain regions at a specific voxel-level, whereas cluster-wise thresholding relies on this decision made about adjacent ‘clusters’ of voxels and is specific to the spatial distribution or size of the clusters. Eklund et al. also examined the in-built auto-correlation functions in the software packages which assign activity to a brain region based on the cluster representing a squared exponential. This is the basic assumption made by the auto-correlation algorithm but in testing this functionality, Eklund et al. found the assumption of spatial smoothness did not follow a Gaussian distribution or was not normally distributed across the entire brain. The lack of spatial smoothness lead the auto-correlation function to incorrectly calculate clusters and in turn, force a false-positive finding.
With the Eklund et al. research actively calling into question the fMRI studies of the past two decades, a heated debate arose around the validity of such a statement and the methods used in the research. Subsequently, the statement was retracted and redefined but this did not go unnoticed. Unfortunately, it does appear that the issue at the heart of this debate has been overlooked and somewhat downplayed which is the matter of reproducibility affecting neuroscience and all science in general. The replication of all results are essential to removing incorrect inferences and misassumptions that lead discoveries to be meaningless without validation. While the debate over the ‘failure’ of fMRI continues to evolve the premise holds that without validation of scientific hypotheses there will never be an opportunity for these to graduate into scientific theories.