Skip to main content
Coronavirus (COVID-19): Latest updates.


Carry On Brexiting

30 January 2017

One final note on this. As I said two posts ago, our Barometer poll asked the following question: “Now that Britain has voted to leave it will need to negotiate the future relationship it has with the European Union. With this in mind, which of the following comes closest to your view?” getting the following pattern of results:


Remain Leave Overall
The priority should be for Britain to have full control of their borders even if it means it is no longer able to trade freely with the EU 17% 77% 47%
The priority should be for Britain to trade freely with the EU even if that means it does not have full control over its borders 53% 8% 26%
Neither of these should be the priority 17% 7% 12%
Don’t Know 13% 8% 15%


However, in December the Western Mail published results from a question that they ran on a Beaufort face-to-face poll conducted in November, which appears to show a somewhat different state of affairs. The Beaufort/Western Mail question asked:

“Looking at this card, please tell me which one of these is more important to you in the negotiations about Brexit (or the UK leaving the European Union)?”


Respondents were then given the following options:


  • Keeping free access to the European Single Market, so that the UK can continue to sell goods and services without incurring tariffs or other obstacles to trade, but means they must allow citizens from all EU countries to work in the UK
  • Restricting the right of people from EU countries to work in the UK, but not having free access to the European Single Market
  • Don’t Know


So we can see that while this question was clearly getting at similar things to the one that we used on our Barometer poll, it was worded with some potentially significant differences. There was only one non-committal response – Don’t Know – while the wording of both the more definite responses in the Beaufort poll was substantially different.


And perhaps because of those differences in wording, the two polls got very different patterns of responses. The table below summarises the patterns across the two samples:


Priority Western Mail/Beaufort Welsh Political Barometer
Single Market Access 54% 26%
Border Controls 30% 47%
Other/Don’t Know 16% 27%


So what do we make of this?! One response might be the standard one of some people, that “all polls are rubbish”. But universal dismissal is no more thoughtful or wise an attitude than universal credulity. A more thoughtful approach might suggest a number of possible factors behind the differences:

  • Probably the least likely candidate is genuine change in attitudes over time. I think this is unlikely for two reasons. First, as a general point, public opinion on such questions normally changes rather more slowly. And second, the findings in our Barometer poll in January were pretty similar to those we found in September. It would be strange indeed for opinion to have shifted radically between September and November, and then shifted right back by early January.
  • Another possible factor, which may explain some of the differences, is sampling variation. This may be the random variation that statistical theory shows we can get from sample to sample, even when everything is conducted perfectly. Or it may be something more systematic, such as differences between the sorts of samples that YouGov and Beaufort are able to conduct using their different approaches.
  • The most likely factor to be accounting for most of the difference, though, is question wording. Simply asking about the issue in these different ways appears to elicit quite different responses.


The best way to test such question wording effects will be to run a split-sample experiment on a future poll – giving a randomly chosen half of the sample one version, and the other half the other version. If the same poll, run at the same time by the same company, produces systematically different results for two different wording of a question, then we can be fairly sure that it was the ‘wording wot done it’.

Talking about such question wording effects may prompt some people to debate what is the ‘best’ or ‘correct’ wording. I’m not sure that it makes much sense to argue in those terms. What I think these findings may point to is the contestability of the issues of market access and border/immigration controls, and their overall place within public debate about the Brexit process. Considered in isolation, both ‘market access’ and ‘border controls’ would probably sound to many people like Good Things. The polling questions are trying to explore what might be peoples’ attitudes if they have to choose between the two of them. But the terms in which such a choice should be presented are not self-evident.

Much of political debate is about the terms in which issues are debated: try to associate your own policies with good things, and associate opponents and their ideas with bad things. And much of the rhetorical debate around Brexit in the coming months will likely be around trying to define the terms through which different options and possibilities are understood. Although there is much about Brexit where many people seem to have fairly fixed views, here at least things still appear to be up for grabs.


  1. Angharad Shaw

    The Western Mail poll spells out more clearly in the question what leaving the single market means. The word ‘trade’ is not one much use in every day speech; it’s usually (these days especially) only heard in a political context.
    When people understand the consequences from the question, they realise that leaving the single market is a bad idea.

  2. J.Jones

    ” all EU countries to work in the UK”

    There is no doubt in my mind that Brexit was driven by a perception of insecure borders across Europe and that it wasn’t EU people coming to “Work” in the UK that was the motivation.

    Beaufort stipulates a particular group…genuine EU citizens coming to employment. People actually thought of all immigrants including those that now sit begging on the streets, high profile, foreign, serious criminals, people from ALL other countries who can be found waiting in A&E on any day. A visible group not the invisible workers.

    Certainly Beaufort’s question encouraged people to think in a more positive way about a certain group…industrious fellow EU citizens. As for the sampling technique; who knows? To a certain degree it is not in the polling companies interest to query their own panel/survey technique too publicly.

Leave a reply

Your email address will not be published. Required fields are marked *