Gatekeeping in Science
16 September 2024It’s common in public discourse to see worries about pseudoscience. This is understandable, for the stakes are high. Although science surely can’t tell us about everything, it is our best way to learn about many important things – not least viruses, climate change, or the health risks of tobacco. Its standards need to be maintained.
But which standards belong to science and which don’t? More generally, which ideas, methods, theories, claims, or even entire disciplines belong and which don’t? And who has the right to make these distinctions? These are questions of gatekeeping in science: determining who and what belongs in or out of science.
Public discourse, including science communication, often comes across as if these questions have obvious answers and so don’t even need to be asked. Words like “scientific”, “anti-scientific”, or “pseudoscientific” are thrown around as if they have clear referents – as if matters of gatekeeping in science are settled.
But they aren’t. Granted, there are general benchmark principles that clearly belong to science. One is that it’s crucial to be open to whatever results your evidence delivers. Another is that claims must be substantiated by contentful arguments, instead of manipulative or biased ones.
But translating these abstract principles into concrete practice is far from straightforward. Take medical research as an example. Most people agree that evidence has an important role to play. But what counts as evidence to begin with – patient testimony? Clinician experience? Randomized controlled trials? Some or all of these? Another thing that’s not clear is which kinds of evidence are stronger than others. When two pieces of evidence point in different directions, for example when quantitative studies conflict with patient testimony or clinician experience, how should we decide what the evidence as a whole says? On top of this, answers to these questions may vary between contexts. Asserting that medical research must be evidence-based is not the end of the conversation, but the beginning.
Answers to questions like these – about how to spell out scientific principles in concrete cases – have important consequences. Scientific credibility matters for government funding or endorsement, for public policy decisions (for example about vaccine mandates), and for many other things. So it’s very important to have a sense of what really is scientific and what isn’t, rather than make indiscriminate claims about misinformation or pseudoscience.
Unfortunately, there is often pressure – from both inside and outside of science – to foreclose too quickly around answers about what is scientific. This is unfortunate because science itself has a measure of unclarity built in. Scientists have to negotiate these questions carefully but open-mindedly on a daily basis. Simple and quick answers are a warning sign that science has gone wrong. So in answer to the question of which standards, methods, and claims belong to science, we may say that part of science’s job is to continue working this out – and never completely stop.
What about the question of who decides what counts as science? On the one hand there are universities, governments, regulatory bodies, and other public or professional institutions. On the other hand, these are made of people, and people are prone to bias, groupthink, special interests, and just plain error. It’s possible that some things may fall short of science’s benchmark principles, yet decision-makers count them as scientific anyway. Or it may be that something respects these principles, yet decision-makers exclude it anyway.
Take for example the committee that wrote a guideline for England, that is taken seriously internationally, on how to treat chronic fatigue syndrome. This committee included many scientists, and its reasoning was supposed to rely on scientific principles. But as it happens, many other scientists – including world experts on chronic fatigue – strongly objected to the guideline and questioned the legitimacy of the committee. They said it let bias and sloppy thinking, while keeping out some treatments that deserved to be kept in. (I’d weigh in firmly with the critics – but that is a conversation for another time.)
This example illustrates that the question of who has the right to decide what counts as scientific is not easy to answer. One thing we can say, at least, is that this too is something that scientists, and perhaps other informed stakeholders, must muddle through as conscientiously as possible.
One lodestar here would be an idea of what gatekeepers in science should be aiming for – what would make a case of gatekeeping good. We would still need to talk about how this idea spells out in concrete cases. But it would at least provide a starting point for conversation.
Together with my co-author Bruce Grimley, I suggest that there are at least three things we can consider in evaluating whether a case of gatekeeping in science is good or bad.
The first is whether the gatekeeping seeks to preserve something that is genuinely valuable, and to keep out things that are not valuable. After all, a good gate is designed to keep the sheep safe from predators. What about science – what valuable thing does good gatekeeping seek to preserve here, and what does it seek to exclude? What it seeks to exclude are things like biased reasoning, fraud, or the interests of career promotion. What it seeks to preserve are the benchmark principles of science. These include – at the very least – the two mentioned above: openness to whatever your evidence delivers, and contentful arguments to substantiate ideas. These make science valuable, and good gatekeeping seeks to preserve them.
But a gatekeeper can seek to preserve something valuable in a dodgy way. For example, an ethics committee might try to keep out undesirables by beating them up. This is dodgy because beating people up is itself unethical. Similarly, gatekeeping in science cannot be particularly good unless it is done in a way that respects the benchmark principles of science. That means that good gatekeeping itself, at the very least, is open to whatever evidence delivers, and engages in contentful argumentation.
But even if gatekeepers seek to preserve scientific principles, and do so in a way that respects them, they might sadly fail. Good gatekeeping in science must therefore be successful. What does successful gatekeeping look like? It preserves the values it sets out to preserve, and keeps out whatever it aims to keep out. The sheep are kept safe and the predators excluded. What does this amount to in science? Successful gatekeeping here preserves the methods, ideas, or hypotheses that really have scientific merit, and it keeps out the ones that don’t.
But this is a narrow tightrope. On the one hand, gatekeeping in science can be too lenient, letting in things that don’t belong. For example, background beliefs from the surrounding culture might slip into what you count as scientific. This happened in the case of the sexist ideology that influenced the doctor Edward Clarke to argue, in respected scientific venues, that higher education would damage women’s fertility.
On the other hand, gatekeeping in science can be excessively tight, keeping out things that deserve a place. This happens, for instance, when unorthodox views are excluded even though they are really on to something and can argue their case. For example, Barbara McClintock’s research on transposition and mobile genetic material was belittled for decades, until it won her a Nobel prize.8 To be fair to overzealous gatekeepers, most unorthodox ideas do fall flat. But some – like McClintock’s – do not, and science can be set back decades by excluding them.
I’ll sum up. Contrary to common misconceptions in public discourse, gatekeeping in science is not straightforward. It is as complex, messy, and in need of conscientious reflection as science itself. There are usually no straightforward answers about which things count as scientific, and about who legitimately decides this. But I’ve suggested three general questions that might be asked to determine what gatekeepers in science should be doing, and to evaluate whether people who act as gatekeepers are doing a good job. These questions are:
- Does the gatekeeping seek to preserve the benchmark principles of science?
- Does the gatekeeping itself abide by these principles?
- Is the gatekeeping successful – neither too lenient, letting in things that don’t belong, nor too tight, keeping out things that deserve consideration?
Of course, spelling out how to answer these in detail, as the example above involving chronic fatigue treatment shows, is a case by case matter. These questions are at least a starting point for a conversation. The hard work in determining what counts as science – or pseudoscience, or bad science – is ongoing. And that is how it should be, because science itself is ongoing.
Picture by the author
- September 2024
- August 2024
- July 2024
- June 2024
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023
- May 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- November 2022
- October 2022
- September 2022
- August 2022
- July 2022
- June 2022
- May 2022
- April 2022
- March 2022
- February 2022
- January 2022
- December 2021
- November 2021
- October 2021
- September 2021
- August 2021
- July 2021
- June 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- December 2020
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
- April 2020
- March 2020
- February 2020
- January 2020
- December 2019
- November 2019
- October 2019
- September 2019
- August 2019
- July 2019
- June 2019
- May 2019
- April 2019
- March 2019
- February 2019
- January 2019
- December 2018
- November 2018
- October 2018
- September 2018
- August 2018
- July 2018
- June 2018
- May 2018
- April 2018
- March 2018
- February 2018
- January 2018
- December 2017
- November 2017
- October 2017
- September 2017
- August 2017
- July 2017
- June 2017
- May 2017