2019 was the third general election since the partnership of ITV-Wales, YouGov and Cardiff University combined to start conducting the Welsh Political Barometer polls in late 2013. Elections provide the hardest of tests for political polling – we have a clear, unambiguous result against which to compare our pre-election polling. So how did we do in 2019?
Our final Welsh Political Barometer poll of the 2019 campaign, published on the Monday before the election, gave the following estimated support levels for each of the parties:
Plaid Cymru 10%
Brexit Party 5%
So how close were we to the final outcome? Now all the Welsh constituencies have declared, we know the final vote share outcomes, and this is what they were:
Plaid Cymru 9.9%
Brexit Party 5.4%
I think even the fiercest critics of polling would have to say that the match between our final Barometer poll and the actual election outcome in Wales is astonishingly close.We should bear in mind that fieldwork for the final Barometer poll was completed three days before the election; we thus had no chance to do anything about any late swings that could have made our poll less accurate. We should also recall that surveys can be expected to have some sampling error: the percentage figures we generate for party support (and on all our other questions) are estimates that we very much hope are close to the true figure, but cannot realistically be expected to be exactly correct. In that context, to get every one of the six largest parties within less than a percentage point of their actual vote share is truly astonishing.
How does this performance compare with the past? The simplest summary measure of the overall performance of a voting intention poll is the Mean Average Error, which pretty much does what it says on the tin: it provides a single measure of how far from the correct outcome the poll was, on average, across all the parties. Here are the mean average errors for our final pre-election poll this year, as well as those for the final pre-election Barometer polls in 2017 and 2015, and the last pre-election poll YouGov did in Wales in 2010:
I would be happy with any of those performances, but that for 2019 is truly exceptional. To average within less than half a single percentage point of the correct figure for all the parties is incredibly close. Huge thanks, and congratulations, go to the fantastically dedicated team at YouGov who do all the hard technical work for our Barometer polls. It has been, and continues to be, a pleasure to work with them.