Functional MRI (fMRI) is 25 years old, yet surprisingly its most common statistical methods have not been validated using real data. Here, we used resting-state fMRI data from 499 healthy controls to conduct 3 million task group analyses. Using this null data with different experimental designs, we estimate the incidence of significant results. In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results.
Oops.
so my cancer detection mri may not have caught what was actually there? And my knee mri may show stuff that is not there? How encouraging!
You would think that false negatives would be the main concern, but false negatives save money on everything from medical care to Social Security, false positives cost money.
Even with a 5% false positive rate, this could be totally worthless for diagnosis, depending on how prevalent the condition is in the first place. A short lesson in Bayesian statistics would teach this to any scientist.
Raymond Tallis, himself a neuroscientist, wrote a detailed critique of imaging in his “Aping Mankind: Neuromania, Darwinitis and the Misrepresentation of Humanity.”
https://www.amazon.com/Aping-Mankind-Neuromania-Darwinitis-Misrepresentation/dp/1844652734
Jesus, Mary and Joseph! I was diagnosed having M>S> from an MRI……WTHell?
MRIs and x-rays are interpretive. My doctors never get to see them anymore.
They just get a written interpretation from the experts whose entire discipline is reading and interpreting these images.
So, “MRI” stands for “Medically, Real Incompetent”…
This article is about FUNCTIONAL MRI (fMRI), which is different than the diagnostic MRIs most people think of, which look at structure rather than function.
fMRI is something I’ve always had my doubts about. A neurologist friend of mine compared it to the early days of phrenology. I ususlly ignore fMRI papers when I’m researching neurology topics unless they have pictures and details of methodology (use of fMRI analysis package means I skip the paper).
If one wants to look at regional brain activation, use SPECT imaging which is a powerful tool and well validated. Assumti9n that data can be fitted with a Gaussian distribution, especially if raw voxel data distributions have fat tails is the height of statistical ignorance.