I ended part two of my recent Electoral Review of 2015 by observing, rather portentiously, that “All that is solid about an opinion poll lead can melt into the electoral air…” I had been reflecting, as you may recall, on how an apparently sizeable opinion poll lead for Labour prior to the 2014 European election had largely disappeared by election day.
Looking back over the opinion poll numbers for that piece, however, set me thinking along slightly different lines. Even after seeing its poll rating edging downwards as the European election approached, Labour still remained some points ahead of its nearest challenger for those elections, UKIP. But on the day, Labour substantially under-performed its final poll rating in the election itself. And as I checked over some other numbers, it became clear that this was not a wholly isolated incident.
Comparing very different polling companies and methods over time can be hazardous. Moreover, it is only since YouGov began fairly regular Welsh polling in late 2009 that we in Wales have consistently had pre-election polls which we can compare to the actual election results. So in the table below I’ve looked only at the performance of YouGov in Wales, compared to the election results, from the 2010 general election onwards. I’ve taken the last pre-election poll prior to each significant electoral contest, and compared the figures produced from the standard voting intention question with the final result. I think the table is quite revealing:
|UK General Election Polls||Lab||Con||Lib-Dem||Plaid||UKIP||Others*|
|ITV-Wales/YouGov, 1-3 May 2010||35||27||23||10||5|
|ELECTION RESULT, MAY 2010||36.2||26.1||20.1||11.3||6.2|
|ITV-Wales/WGC/YouGov, May 2015||38||26||7||12||13||4|
|ELECTION RESULT, MAY 2015||36.9||27.2||6.5||12.1||13.6||3.7|
|EP Election Polls|
|ITV-Wales/WGC/YouGov, May 2014||33||16||7||15||23||7|
|ELECTION RESULT, MAY 2014||28.15||17.4||3.95||15.3||27.55||7.6|
|WGC/YouGov, May 2012||48||17||7||14||15|
|ELECTION RESULT, MAY 2012#||42||13||8||14||22|
|NAW Constituency Vote|
|ITV-Wales/YouGov, 2-4 May 2011||47||20||9||18||6|
|ELECTION RESULT, MAY 2011||42.3||25.0||10.6||19.3||2.8|
|NAW List Vote|
|ITV-Wales/YouGov, 2-4 May 2011||43||19||8||18||13|
|ELECTION RESULT, MAY 2011||36.9||22.5||8.0||17.9||14.7|
*Including UKIP until 2014 #Local elections covered all of Wales except for Ynys Môn.
Overall, the performance of these final polls was pretty good, without being flawless. As one would probably expect, the polls were more accurate in the highest turnout elections – the two general elections, where there was a mean average error per party of 1.5% in 2010 and 0.82% in 2015. (Rather ironically, it was in 2015 – supposedly the disastrous year for the pollsters – that YouGov’s Welsh polls were the most accurate of all the six ballots observed here!) The final poll was least accurate in the lowest turnout election, the 2014 European Parliament one, where there was a mean error per party of 4.31%.
But where did these errors come from? There are some fascinating differences that emerge when we compare the parties and how their electoral performance relates to the final opinion polls. Labour has under-performed its poll rating in all of the elections except that of 2010; for the Conservatives, the exact opposite was the case apart from the 2012 local elections. (And local elections may be a special case for the Conservatives, given the strong continuing tradition of small-c conservative Independent councillors in much of Wales). Plaid Cymru’s support has thus far always been pretty accurately estimated by YouGov, who twice have been within 0.1 percentage points of their vote share, and never more than 1.3% out; however, on four out of the six instances Plaid have been (slightly) under-estimated. The Liberal Democrats’ support has been over-estimated slightly more often than not, while on both occasions since they emerged as a major party UKIP’s vote share has been under-stated by YouGov – although the extent of the under-statement was much smaller in the general election than in 2014’s European election.
That 2010 was the one election in which Labour support was under-stated, and Conservative support over-stated outside of local elections, could well be more than just a coincidence. Before the May 2015 general election, Matt Singh of Number Cruncher Politics conducted an extensive review of the performance of the GB-wide polls against final election vote shares over recent decades. This review enabled Matt to be just about the only person I know of who publicly predicted, before the fact, the errors made by the opinion polls in last year’s general election. (Indeed, Matt not only predicted that the polls would get things wrong, but predicted almost precisely the magnitude of their error. And he apparently backed this up with some bets, so making money out of it!). Matt suggested that 2010 was something of an outlier to much longer-standing patterns; perhaps because of the turbulence caused by the mid-campaign eruption of ‘Clegg-mania’, it was atypical to the normal relationships between opinion polls and general elections outcomes, patterns which emphatically re-asserted themselves in 2015.
The 2010 general election aside, Labour’s recent record in Wales has been one of under-performing its opinion poll rating in elections. Moreover, that tendency appears more marked in lower-turnout contests like the Assembly and European elections. The Conservatives, by contrast, tend to do better than the polls suggest.
I’m certainly not saying that this is some sort of immutable, iron law of Welsh electoral politics. We don’t have nearly enough data points to indicate that. Moreover, recent changes by YouGov to their methodology – which should be rolled out in full for the first time in Wales with our next Barometer poll – may help to reduce the disparities we observe here.
What I am saying is that the evidence seems to suggest that Welsh Labour have been having some problems getting their vote out in recent years; moreover, those problems seem to have been of greater magnitude in those elections that generally attract a lower voter turnout. Unless Labour can successfully address that problem before May, this year’s National Assembly election might just turn out to be a little more competitive than the polls would currently seem to indicate is probable.