We’ve witnessed in recent months a good deal of high-profile debate over the scientific method and the results it produces. The grappling over established research on seemingly everything from vaccines and chronic disease to diet and exercise certainly feeds that portion of the population that thrives on controversy, but there’s nothing particularly novel or controversial about questioning the veracity of a study. Scientists, like the rest of us, are fallible humans, and their work has always been vulnerable to scrutiny.
“The causes of error are multitudinous, and range from carelessness to dishonesty, from wishful thinking to outright corruption, with everything in between,” writes Theodore Dalrymple, MD, in his 2019 book, False Positive. “Medical research relies on ever-more-complex statistics to conclude anything at all, and common sense is often the first victim of technical sophistication. But there is no such thing as a perfect paper that omits nothing, answers all possible questions, and cannot be criticized on any ground.”
Dalrymple arrived at this conclusion after reviewing a year’s worth of research papers published in the venerable New England Journal of Medicine. More recently, an ambitious collaboration of 865 researchers reached a similar verdict after reviewing 3,900 papers published in 62 professional journals covering a wide range of topics. Their findings, published last week in the journal Nature, demonstrate just how rarely existing research results can be replicated.
The researchers, working under the auspices of the Systematizing Confidence in Open Research and Evidence (SCORE) program, set out to analyze the original studies and determine whether they could use the same data to produce the same results. But they struggled to access the necessary information and, when they did, often reached a different conclusion. In as many as a third of the studies researchers reviewed, in fact, attempts to replicate the published results were unsuccessful.
“The main message of SCORE is a simple one: research is hard. And, in some ways, the hard work begins after making a discovery,” notes Tim Errington, PhD, senior director of research at the Center for Open Science and one of the project’s leaders. “A tremendous amount of effort is needed to verify and have enough confidence in new discoveries to build foundations for further discovery.”
“A tremendous amount of effort is needed to verify and have enough confidence in new discoveries to build foundations for further discovery.”
The SCORE report only generalized the specific lessons offered a few days earlier, when I had noticed a couple of classic cases of repudiated pearls of scientific wisdom.
Researchers for years have warned us to avoid too much sedentary behavior, as it has been associated with the same health risks as smoking. A study published last year in Alzheimer’s & Dementia, for instance, even found a link between too much sitting around and a higher risk of developing dementia over a seven-year follow-up period.
So, what are we to make of a report published last month in the American Journal of Preventive Medicine that suggests persistent sitting may actually reduce the risk of dementia?
The study, led by Mats Hallgren, PhD, of Sweden’s Karolinska Institutet, analyzed self-reported sedentary patterns among more than 20,000 people between the ages of 35 and 64 who participated in the Swedish National March Cohort. Hallgren’s team tracked incidences of dementia among those individuals from a national healthcare database over a 19-year follow-up period. Their conclusion: Each additional hour of “mentally active sedentary behavior” — office work, for example, rather than watching TV — during the day was associated with a 4 percent reduced risk of developing the disease.
“Sedentary behavior is a ubiquitous but modifiable risk factor for many health conditions, including dementia,” Hallgren says. “Our study adds the observation that not all sedentary behaviors are equivalent; some may increase the risk of dementia, while other may be protective. It is important to remain physically active as we age, but also mentally active — especially when we are sitting.”
The other case involves the so-called placebo effect. Placebos have long been viewed as medicinal deception involving some sham antidote patients are led to believe will remedy their illness. The results of a new study argue, however, that a placebo may do the trick even when the patient knows it’s just a placebo.
Italian researchers recruited 90 healthy older adults and assigned them to one of three groups: a control group that received no medication, one that was given a placebo but told it was a drug designed to improve overall functioning, and a third group that knew they were taking a placebo but were told it may offer some mind-body benefits. Unlike the second group, which received a traditional placebo approach, the third group was given what researchers call an “open-label placebo” — they were told the pill was inert. During the three-week study period, participants reported levels of stress, fatigue, and other aspects of well-being. At the conclusion of the study, they underwent tests measuring short-term memory, attention, and physical performance.
Researchers found that participants in the third group improved their cognitive performance by a larger margin than members of the other two groups. “Overall, the findings show that placebo interventions were associated with modest improvements in perceived stress, cognitive performance, and psychological functioning, with open-label placebos matching or surpassing the effects of deceptive placebos,” the authors write in the International Journal of Clinical and Health Psychology.
It was a rather small sample size, however, and the results don’t really “constitute direct evidence of treatment efficacy,” they admit. And . . . wait for it: “[T]he present results should be considered preliminary.”
This Post Has 0 Comments