Gatekeeping in Science

It’s common in public discourse to see worries about pseudoscience. This is understandable, for the stakes are high. Although science surely can’t tell us about everything, it is our best way to learn about many important things – not least viruses, climate change, or the health risks of tobacco. Its standards need to be maintained.

But which standards belong to science and which don’t? More generally, which ideas, methods, theories, claims, or even entire disciplines belong and which don’t? And who has the right to make these distinctions? These are questions of gatekeeping in science: determining who and what belongs in or out of science.

Public discourse, including science communication, often comes across as if these questions have obvious answers and so don’t even need to be asked. Words like “scientific”, “anti-scientific”, or “pseudoscientific” are thrown around as if they have clear referents – as if matters of gatekeeping in science are settled.

But they aren’t. Granted, there are general benchmark principles that clearly belong to science. One is that it’s crucial to be open to whatever results your evidence delivers. Another is that claims must be substantiated by contentful arguments, instead of manipulative or biased ones.

But translating these abstract principles into concrete practice is far from straightforward. Take medical research as an example. Most people agree that evidence has an important role to play. But what counts as evidence to begin with – patient testimony? Clinician experience? Randomized controlled trials? Some or all of these? Another thing that’s not clear is which kinds of evidence are stronger than others. When two pieces of evidence point in different directions, for example when quantitative studies conflict with patient testimony or clinician experience, how should we decide what the evidence as a whole says? On top of this, answers to these questions may vary between contexts. Asserting that medical research must be evidence-based is not the end of the conversation, but the beginning.

Answers to questions like these – about how to spell out scientific principles in concrete cases – have important consequences. Scientific credibility matters for government funding or endorsement, for public policy decisions (for example about vaccine mandates), and for many other things. So it’s very important to have a sense of what really is scientific and what isn’t, rather than make indiscriminate claims about misinformation or pseudoscience.

Unfortunately, there is often pressure – from both inside and outside of science – to foreclose too quickly around answers about what is scientific. This is unfortunate because science itself has a measure of unclarity built in. Scientists have to negotiate these questions carefully but open-mindedly on a daily basis. Simple and quick answers are a warning sign that science has gone wrong. So in answer to the question of which standards, methods, and claims belong to science, we may say that part of science’s job is to continue working this out – and never completely stop.

What about the question of who decides what counts as science? On the one hand there are universities, governments, regulatory bodies, and other public or professional institutions. On the other hand, these are made of people, and people are prone to bias, groupthink, special interests, and just plain error. It’s possible that some things may fall short of science’s benchmark principles, yet decision-makers count them as scientific anyway. Or it may be that something respects these principles, yet decision-makers exclude it anyway.

Take for example the committee that wrote a guideline for England, that is taken seriously internationally, on how to treat chronic fatigue syndrome. This committee included many scientists, and its reasoning was supposed to rely on scientific principles. But as it happens, many other scientists – including world experts on chronic fatigue – strongly objected to the guideline and questioned the legitimacy of the committee. They said it let bias and sloppy thinking, while keeping out some treatments that deserved to be kept in. (I’d weigh in firmly with the critics – but that is a conversation for another time.)

This example illustrates that the question of who has the right to decide what counts as scientific is not easy to answer. One thing we can say, at least, is that this too is something that scientists, and perhaps other informed stakeholders, must muddle through as conscientiously as possible.

One lodestar here would be an idea of what gatekeepers in science should be aiming for – what would make a case of gatekeeping good. We would still need to talk about how this idea spells out in concrete cases. But it would at least provide a starting point for conversation.

Together with my co-author Bruce Grimley, I suggest that there are at least three things we can consider in evaluating whether a case of gatekeeping in science is good or bad.

The first is whether the gatekeeping seeks to preserve something that is genuinely valuable, and to keep out things that are not valuable. After all, a good gate is designed to keep the sheep safe from predators. What about science – what valuable thing does good gatekeeping seek to preserve here, and what does it seek to exclude? What it seeks to exclude are things like biased reasoning, fraud, or the interests of career promotion. What it seeks to preserve are the benchmark principles of science. These include – at the very least – the two mentioned above: openness to whatever your evidence delivers, and contentful arguments to substantiate ideas. These make science valuable, and good gatekeeping seeks to preserve them.

But a gatekeeper can seek to preserve something valuable in a dodgy way. For example, an ethics committee might try to keep out undesirables by beating them up. This is dodgy because beating people up is itself unethical. Similarly, gatekeeping in science cannot be particularly good unless it is done in a way that respects the benchmark principles of science. That means that good gatekeeping itself, at the very least, is open to whatever evidence delivers, and engages in contentful argumentation.

But even if gatekeepers seek to preserve scientific principles, and do so in a way that respects them, they might sadly fail. Good gatekeeping in science must therefore be successful. What does successful gatekeeping look like? It preserves the values it sets out to preserve, and keeps out whatever it aims to keep out. The sheep are kept safe and the predators excluded. What does this amount to in science? Successful gatekeeping here preserves the methods, ideas, or hypotheses that really have scientific merit, and it keeps out the ones that don’t.

But this is a narrow tightrope. On the one hand, gatekeeping in science can be too lenient, letting in things that don’t belong. For example, background beliefs from the surrounding culture might slip into what you count as scientific. This happened in the case of the sexist ideology that influenced the doctor Edward Clarke to argue, in respected scientific venues, that higher education would damage women’s fertility.

On the other hand, gatekeeping in science can be excessively tight, keeping out things that deserve a place. This happens, for instance, when unorthodox views are excluded even though they are really on to something and can argue their case. For example, Barbara McClintock’s research on transposition and mobile genetic material was belittled for decades, until it won her a Nobel prize.⁸ To be fair to overzealous gatekeepers, most unorthodox ideas do fall flat. But some – like McClintock’s – do not, and science can be set back decades by excluding them.

I’ll sum up. Contrary to common misconceptions in public discourse, gatekeeping in science is not straightforward. It is as complex, messy, and in need of conscientious reflection as science itself. There are usually no straightforward answers about which things count as scientific, and about who legitimately decides this. But I’ve suggested three general questions that might be asked to determine what gatekeepers in science should be doing, and to evaluate whether people who act as gatekeepers are doing a good job. These questions are:

1. Does the gatekeeping seek to preserve the benchmark principles of science?

2. Does the gatekeeping itself abide by these principles?

3. Is the gatekeeping successful – neither too lenient, letting in things that don’t belong, nor too tight, keeping out things that deserve consideration?

Of course, spelling out how to answer these in detail, as the example above involving chronic fatigue treatment shows, is a case by case matter. These questions are at least a starting point for a conversation. The hard work in determining what counts as science – or pseudoscience, or bad science – is ongoing. And that is how it should be, because science itself is ongoing.

Picture by the author