Uncategorized

All That is Solid…Sometimes Stays Solid

A few months prior to the Assembly election I published a piece which looked at the performance of YouGov polls in Wales in comparison to election results.

http://blogs.cardiff.ac.uk/electionsinwales/2016/01/18/all-that-is-solid-2/

As I observed, during the years since YouGov started polling regularly in Wales, we have had several opportunities to compare their final pre-election vote intention figures with the actual results. The general pattern, it appeared, has been for the Labour party to do less well in elections than in the final poll – and particularly so in lower turnout elections. By contrast, the Conservative party seemed mostly to have outperformed their final poll ratings in Wales in recent years.

I’ve already discussed on the blog how our innovative ‘on-the-day’ poll performed at the Assembly election. But the more normal method of evaluating the accuracy of opinion polls in relation to elections is to look at the final pre-election poll. The table below therefore shows the numbers produced for all main parties on both ballots by YouGov’s ‘final call’ poll (published on the evening of Wednesday 4th May, with the fieldwork conducted on 2-4 May), in comparison with the actual election result.

 

  Labour Cons LibDem Plaid UKIP Others
Final Poll: Constituency 33 21 8 19 16 4
CONSTITUENCY RESULT 34.7 21.1 7.7 20.5 12.5 3.4
Final Poll: Region 31 20 6 20 16 8
REGION RESULT 31.5 18.8 6.5 20.8 13.0 9.5

 

I think we can say from these figures that the final call poll, just like the on-the-day poll, performed very strongly. The mean average error across all the parties was only 1.28 percentage points for the constituency ballot and 1.25 points for the regional ballot. That is a very high degree of accuracy. Moreover, unlike with the GB-wide opinion polls last year, the poll also got all the main ‘narrative’ issues broadly right: Labour were well in the lead, the Conservatives and Plaid were competing very closely for second place, UKIP were in a strong fourth, and the Liberal Democrats in a distant fifth.

However, no poll is ever perfect, and it is interesting to look at those errors in estimation which did occur. Let’s take them in turn.

Labour: For the first time since 2010, a YouGov pre-election poll in Wales actually under-estimated Labour’s vote share in an election. This under-estimation occurred on both the constituency and regional ballots. However, the magnitude of the error was small: 1.7 points on the constituency vote and a mere 0.5 points for the regional one.

Why was the previous tendency to over-state Labour support not evident this year? There are a number of possibilities:

  • One possibility might be that the final call poll simply happened to produce a particular low estimate of Labour support because of a sample that was unfavourable to Labour. But this seems wrong: the final call poll saw Labour constituency vote intention unchanged from the previous poll, and actually gave Labour a two-point increase on the regional vote. The final call poll was definitely not an ‘outlier’ from the general picture of Labour support shown in other recent YouGov polls. So I think we can rule out this potential explanation.
  • A second possibility is that the changes made by YouGov to their methodology in recent years – including small changes after the 2014 European election, and the rather greater ones made after the 2015 general election – have simply led to lower, and generally more accurate, estimates of Labour support. Indeed, if this explanation is valid then, if anything, YouGov may have slightly ‘overshot’ in these changes.
  • A third possibility, not entirely inconsistent with the second, is that Labour’s campaign may have been more effective this year than in some other recent elections – either at shifting some last-minute undecided voters to support Labour, or at mobilising their existing support.

 

Conservative: In contrast to Labour, in previous years YouGov polls in Wales have tended to under-estimate Conservative electoral support. That was not the case this year: the constituency vote share was estimated almost exactly correctly, while the regional vote share was over-estimated by 1.2 points. Again, it may well be that methodological changes have eliminated previous errors. It could also be, though, that the Conservatives were less effective in 2016 at persuading people and mobilising support than in previous elections.

 

Plaid Cymru: Plaid support was slightly under-estimated by the final call poll. But this was only by a small margin on both ballots. Moreover, the on-the day poll actually got Plaid’s regional vote support almost exactly correct. This suggests that, as with previous elections, there were no major systematic problems in estimating Plaid Cymru vote share.

 

Liberal Democrats: Their support was estimated very accurately on both ballots.

 

UKIP: UKIP were the biggest source of error in the final call poll, which over-estimated their actual vote share by 3.0 points on the regional ballot and 3.5 points on the constituency one. This came as no great shock to me. Indeed, I had suggested prior to the election two reasons why we might expect UKIP’s vote share in the Assembly election to be somewhat below that which the Yougov polls were suggesting:

  • First, YouGov’s GB-wide polls were estimating UKIP support at rather higher levels than most other companies; it was therefore at least possible that youGov might be overstating UKIP support;
  • Second, the Welsh Election Study pre-election wave had found UKIP supporters to be rather less interested in the Assembly election than were supporters of several other parties. It was therefore possible that UKIP supporters might be less inclined to turn out and vote.

 

The extent to which either of these points are valid explanations of UKIP’s under-performance is something which YouGov will doubtless be pondering. But any minor re-calibration of their methodology after the Assembly election may produce adjustments that tend slightly to reduce estimates of UKIP support in Wales.

 

Endnote: Here is the up-dated full table of all YouGov pre-election polls since 2010, alongside the actual election result.

 

UK General Election Polls Lab Con LibDem Plaid UKIP Others*
May 2010 35 27 23 10   5
2010 RESULT 36.2 26.1 20.1 11.3   6.2
 

May 2015

 

38

 

26

 

7

 

12

 

13

 

4

2015 RESULT 36.9 27.2 6.5 12.1 13.6 3.7
 

NAW Polls: Constit. Vote

           
May 2011 47 20 9 18   6
2011 RESULT 42.3 25.0 10.6 19.3   2.8
 

May 2016

 

33

 

21

 

8

 

19

 

16

 

4

2016 RESULT 34.7 21.1 7.7 20.5 12.5 3.4
 

NAW Polls: Regional Vote

 

43

 

19

 

8

 

18

   

13

May 2011 36.9 22.5 8.0 17.9   14.7
 

May 2016

 

31

 

20

 

6

 

20

 

16

 

8

2016 RESULT 31.5 18.8 6.5 20.8 13.0 9.5
 

2014 EP Election Poll

 

33

 

16

 

7

 

15

 

23

 

7

2014 RESULT 28.15 17.4 3.95 15.3 27.55 7.6
 

2012 Local Elects Poll

 

48

 

17

 

7

 

14

   

15

2012 RESULT# 42 13 8 14   22

*Including UKIP until 2014; #Local elections covered all of Wales except for Ynys Môn.

Comments

  • Jim Shepherd Foster

    Over time the Welsh Assembly is being seen as an arms length issue for the voters supporting main parties. UKIP are still very much the protest vote and as such doesn’t command loyalty, (more important in Wales). In my campaigning days (2010) switching between parties was not generally seen as an option, ‘no show’ was the preferred register of disapproval rather than switching parties. That UKIP did so well should be an alarm call to the main parties. The Assembly elections taking place so close to, but before the EU referendum, also depressed the UKIP potential vote. Whatever the referendum result UKIP would have done better in Assembly elections post the referendum. (Scottish effect)

  • John Thorne

    Far from being an “arms length” issue the Assembly is increasingly recognised as directly pertinent to Welsh life and this is reflected in a slow but steady increase in participation, whereas the Westminster voting system and tribal pattern of governance is seen as increasingly remote. I would have thought that the elections being so close to the EU referendum would have increased rather than decreased the UKIP vote. I don’t think that the “Scottish effect” is in any way similar to what may happen to UKIP support following a Remain vote. The party is now, as it has always been, primarily a party of protest in whatever electoral context. An on-going but marginalised presence as a party of general disaffection is the most probable fate of UKIP especially now that their presence in the Senedd brings their underlying right wing philosophy into sharper focus.

Leave a Reply

Your email address will not be published. Required fields are marked *