Science is losing its ability to disrupt

The writer is a scientific commentator

There is, according to an eye-catching metric doing the rounds, a malaise at the heart of the scientific enterprise. This noble pursuit seems to be slowing down in its ability to disrupt convention.

Evidence of that deceleration, according to the metric’s creators, can be seen in decreasingly novel patent applications. Rather than minting revolutionary ways of thinking, science and technology are increasingly polishing the same conceptual pennies. If iterative research is displacing its more radical cousin, to the possible detriment of knowledge, human well-being and the economy, then science and technology may require some disruption of their own.

Russell Funk, associate professor in strategic management and entrepreneurship at Minnesota University, teamed up with PhD student Michael Park and Erin Leahey, a sociology professor at Arizona University, to analyze 45mn scientific papers and 3.9mn patents. They gave each paper and patent a “consolidation-disruption index” based on whether it was built on previous findings or sent a field in a new direction.

Their intuition, the researchers wrote in the scientific journal Nature this month, was that “if a paper or patent is disruptive, the subsequent work that cites it is less likely to also cite its predecessors”. For example, the first paper to announce that the Earth goes round the Sun would be referenced frequently by future scholars — but not the many earlier ones that (incorrectly) assert the opposite. The CD index ranged from -1 for a consolidating paper, to 1 for a disruptive paper.

In several fields, the average CD index became less positive as the decades rolled by, suggesting a decline in disruptiveness. In the physical sciences, the CD index dropped from 0.36 to zero between 1945 and 2010. Patents also showed a decline: in “computers and communications”, the index fell from 0.30 to 0.06 between 1980 and 2010; patents in the “drugs and medicine” category slipped from 0.38 to 0.03. Social sciences declined the most; life sciences and biomedicine the least.

Why are science and technology apparently becoming less groundbreaking? The US science writer John Horgan, who penned The End of Science in 1996, believes the era of blockbuster discoveries is over. “It dovetails with my analysis that science is hitting a wall,” he tells me, adding that he finds no pleasure in vindication. “I want to see revelation after revelation — forever! But sadly, that’s not the way scientific discovery works.” Horgan argues there are only so many truths to uncover, only so many paradigms to shift — and thus a finite pot of disruption to exhaust.

The researchers are less convinced by this “low-hanging fruit” thesis, saying it is unlikely that multiple disciplines would be demystified to the same extent simultaneously. Rather, the snowballing of scientific output, they suggest, leads to researchers mastering ever-narrower domains of knowledge. That funneling limits the scope for novel ideas that stimulate new industries. It is not implausible to imagine a connection between this funneling and, say, the productivity crisis in pharmaceutical R&D, or the “Great Stagnation” described by economist Tyler Cowen.

Tim Minshall, professor of innovation at Cambridge University, believes the decline in disruptive research is worrying for humanity. He thinks the whole system deserves scrutiny: from the incentives and barriers faced by individuals, universities, companies and governments undertaking risky research, to the challenges of scaling up radical ideas to capture their full social and economic value. But he cautions that “disruptive ideas alone do not lead to progress. . . we need the incremental research that stretches and tests the disruptive ideas”.

Others have noted the fall-off in company-led research since the 1970s: work at corporate research arms such as Bell Labs and DuPont, for example, produced both commercial products and Nobel Prizes. Today, with the exception of behemoths like Meta and Google, most research happens in universities, where a “publish or perish” culture largely rewards quantity over quality, and the incremental over the radical.

Compared with 60 years ago, there are more researchers in more institutions who are publishing more papers in more journals. That proliferation, according to a 2021 study that noted a similar funneling, was leading to an “ossification of canon” and also making it tougher to pick out the breakthroughs that matter. “Publish or perish” cannot be the only research mantra. We need a well-supported philosophy of “disrupt or ossify”, too.

Leave a Reply

Your email address will not be published. Required fields are marked *