The Replication Crisis in Medicine—An Era of Digital Growth

COURTENEY MALIN – Since the mid-1900s, increasing access to digital technology has allowed scientists to conduct research at a rapid pace. The first reproducibility crisis began in psychology in the 1980s and 1990s when well-known studies taught in psychology textbooks around the world could not be replicated in modern experiments. Long-standing concepts that were previously known to apply to all humans behaviourally, cognitively, and developmentally fell apart in a few months, leading researchers and the general public to question what other information could have escaped the rigorous review that is the foundation of experimentation and research.

When the replication crisis began in psychology, many scientists and researchers lacked concern since psychology is a social science that had been rooted heavily in correlational research rather than true experimentation. Even psychologists were not concerned, citing old instrumentation and a lack of resources as confounding variables in these studies. Yet this collapse of fundamental research also started to change how young researchers approached experimentation. Science has always been driven by the desire to discover and expand knowledge, but this desire also led many scientists away from the unfashionable field of replicating studies. Most scientists want to discover rather than re-analyze old data—in essence, the preference is to build another floor on a skyscraper rather than repetitively check the base for stability.

However, more and more scientists are dedicating their time to replicating studies, resulting in more information about the integrity of the initial findings. Even large, modern studies have been subject to criticism for skewed or unlikely results.

Statistics is considered one of the main contributors to a lack of replicability. The level of significance varies based on the data, and it is not a value that is “set in stone.” Rather, it changes with new information, the number of participants within a study, and whether a placebo is used as a control. Depending on the content and design of a study, it can also change how this level of significance is interpreted. For example, as mentioned above, a study testing a medication could show a higher level of treatment significance if it does not have a placebo control group and if it is compared solely to a group without the medication. The placebo effect may account for most of the effects of the drug; however, the statistical analysis makes it appear that the drug is extremely effective.

In addition, scientists can be pressured to acquire valuable data and results. Since science funding comes primarily from universities/ institutions, industry, and private donors, a scientist may need some conclusive data to maintain funding. Statistics can be easily manipulated to create these results. Participants can be dropped, data altered, and results misconstrued. In addition, until recently with digital reproductions, it was very costly to reproduce a study. Many scientists who were able to receive funding would not want to “waste” their funding on reproducing another researcher’s study. They would prefer to conduct their own innovative research. Therefore, the source of science funding has contributed to this crisis.

Lastly, the guardians of published science, peer reviewers, have increased the number of poor studies circulating in scientific literature. How often are studies with no correlation or negative findings published? Rarely. Scientists tend to not submit null findings since they are not attention-grabbing and peer reviewers have no incentive to publish them. People gravitate toward positive findings and interesting results. This system allows false-positives into scientific literature and eventually out into the public where the ideas will take hold. Certain fads such as coffee bean tablets for weight loss, turmeric pills to curb-cravings, or soybean oil to prevent breast cancer can all be traced to poor studies with insignificant results that were never replicated, and therefore, caught the eye of the media.

While this replication crisis may be unsettling, it has enabled scientists and researchers to reestablish the foundation of science. Science depends on replicating studies to further support, qualify, or contradict the results of previous studies. Since experiments can never be perfect, this system allows for theories to change and reflect new knowledge. The replication crisis has shown that there is a necessity for the reproduction of studies, leading researchers into an era of more reliable science.

Photography Source: https://www.eofproject.net/blog/replication-crisis-the-disgraced-emotionalization-of-science-the-publish-or-perish-infection