As brain scans have become more detailed and informative in recent decades, neuroimaging has seemed to promise a way for doctors and scientists to “see” what’s going wrong inside the brains of people with mental illnesses or neurological conditions. Such imaging has revealed correlations between brain anatomy or function and illness, suggesting potential new ways to diagnose and treat psychiatric, psychological and neurological conditions. But the promise has yet to turn into reality, and a new study explains why: The results of most studies are unreliable because they involved too few participants.
Scientists rely on brainwide association studies to measure brain structure and function — using MRI brain scans — and link them to complex characteristics such as personality, behavior, cognition, neurological conditions, and mental illness. But a study by researchers at Washington University School of Medicine in St. Louis and the University of Minnesota, published March 16 in Nature, shows that most published brainwide association studies are performed with too few participants to yield reliable findings.
Using publicly available data sets – involving a total of nearly 50,000 participants – the researchers analyzed a range of sample sizes and found that brainwide association studies need thousands of individuals to achieve higher reproducibility. Typical brainwide association studies enroll just a couple dozen people.
Such so-called underpowered studies are susceptible to uncovering strong but spurious associations by chance while missing real but weaker associations. Routinely underpowered brainwide association studies result in a glut of astonishingly strong yet irreproducible findings that slow progress toward understanding how the brain works, the researchers said.
“Our findings reflect a systemic, structural problem with studies that are designed to find correlations between two complex things, such as the brain and behavior,” said senior author Nico Dosenbach, MD, PhD, an associate professor of neurology at Washington University. “It’s not a problem with any individual researcher or study. It’s not even unique to neuroimaging. The field of genomics discovered a similar problem about a decade ago with genomic data and took steps to address it. The NIH (National Institutes of Health) began funding larger data-collection efforts and mandating that data must be shared publicly, which reduces bias, and as a result, genome science has gotten much better. Sometimes you just have to change the research paradigm. Genomics has shown us the way.”