Uncategorized

So How Did the Ashcroft Polls Do In Wales?

One of the innovations seen in the period prior to the 2015 general election was the large number of constituency polls published – nearly all of them by Lord Ashcroft. We had quite often seen ‘marginals’ polls reported previously, but those were usually done with a standard sample size spread over a number of marginal seats – and thus having only a small number of respondents in each constituency. What was new about Lord Ashcroft’s polls was full-scale samples done in a large number of individual seats.

Another innovation of Lord Ashcroft’s polls was to ask, in all those conducted, two different voting intention questions. The first one was a standard, ‘generic’ question:

‘If there was a general election tomorrow, which party would you vote for?’

 

This was then followed by a question which specifically prompted respondents to think about the particular dynamics in their own seat:

‘Thinking specifically about your own parliamentary constituency at the next general election and the candidates who are likely to stand for election to Westminster there, which party’s candidate do you think you will vote for in your own constituency?’

 

The responses to this latter, constituency-specific question were generally used as the ‘headline’ figures reported from the polls. Now that the election is over, we can evaluate how these constituency polls, and the two voting intention questions contained within them, performed in relation to the actual election results across the five Welsh seats that Lord Ashcroft polled – Cardiff North, Cardiff Central, Brecon & Radnor, Carmarthen West & South Pembrokeshire, and the Vale of Glamorgan.

Before we do so, one word of caution should be entered. A poll is always an attempt to measure opinion at the time it was taken: in the phrase Lord Ashcroft himself likes to use, it is ‘a snapshot, not a prediction’. Some of the Ashcroft constituency polls were done quite some time before the election, and the further away from the election a poll is taken the less reliable we should expect it to be as a guide for how the election actually turns out. Please bear that in mind.

That qualifier entered, below I present five tables, for each of the five seats polled. (For Cardiff North, which was polled twice by Lord Ashcroft, we’ll take the more recent of the two polls). For each party the table will list three key pieces of information:

  • That party’s share on the ‘generic’ voting intention question
  • That party’s share on the constituency-specific voting intention question
  • Their actual 2015 general election result in the seat

The tables also list, at the bottom of the ‘generic’ and ‘constituency-specific’ columns, the overall mean error: how much, on average, the polling figures differed from the actual election result per party.

 

Cardiff North (July 2014)

Party Generic Constituency Result
Conservative 31 30 42.4
Labour 38 41 38.3
UKIP 14 12 7.7
Plaid Cymru 5 7 4.5
Lib-Dems 5 6 3.8
Greens 3 3 2.5
MEAN ERROR 3.37 4.10

 

Cardiff Central (September 2014)

Party Generic Constituency Result
Labour 37 36 40.0
Lib-Dem 16 24 27.1
Conservative 22 19 14.7
UKIP 10 9 6.5
Greens 7 5 6.4
Plaid Cymru 6 9 5.0
MEAN ERROR 4.42 3.22

 

Brecon & Radnor (November 2014)

Party Generic Constituency Result
Conservative 32 27 41.1
Lib-Dems 18 31 28.3
Labour 18 15 14.7
UKIP 21 17 8.3
Plaid Cymru 7 8 4.4
Greens 4 2 3.1
MEAN ERROR 6.48 5.08

 

Carmarthen West & South Pembs (Dec 2014)

Party Generic Constituency Result
Conservative 32 33 43.7
Labour 30 29 28.7
UKIP 16 14 11.6
Plaid Cymru 13 16 10.4
Greens 3 3 3.2
Lib-Dems 4 4 2.4
MEAN ERROR 3.63 3.47

 

Vale of Glamorgan (February 2015)

Party Generic Constituency Result
Conservative 40 38 46.0
Labour 32 32 32.6
UKIP 10 10 10.7
Plaid Cymru 8 12 5.6
Lib-Dems 5 4 2.6
Greens 5 3 2.1
MEAN ERROR 2.5 3.0

 

 

So what can we make of all that? A few observations:

  • When comparing the elections outcomes to the results of the two voting intention questions, the constituency-specific question was, overall, closer to the result in three of the seats (albeit only very narrowly, by 0.16% in one instance) while the generic question was closer in the other two. In four out of the five seats (all except Cardiff North) the generic question ‘predicted’ the correct winner; the constituency-specific question got it right in three out of five (incorrectly ‘predicting’ the Liberal Democrats to win in Brecon & Radnor). Certainly in Wales there was no demonstrable superiority to either form of question.
  • When we look at specific parties, we do find that the constituency-specific question was closer to the correct result for the Liberal Democrats in the two examples where they were defending their own seats. But for Plaid Cymru, the generic question was closer to the final result in all five constituencies.
  • There was also no very obvious trend for the Ashcroft polls to be closer to the actual result the closer they were conducted to the election.

Overall, were these constituency polls worth it? Perhaps that depends on whether you have a few tens of millions of pounds going spare… We await a full analysis of all the Ashcroft constituency polls across Britain. But the experience of them in Wales does suggest that they were certainly not exempt from some of the problems that faced the opinion polls in 2015.

Postscript (02/06/15): Just a brief addition to this post, in response to some comments both here and on Twitter, about the performance of Plaid Cymru in these polls. The first point is a reminder that all the Ashcroft constituency polls were conducted by telephone. (There is a very good reason for this: even the company with the largest on-line panel, YouGov, would struggle to find samples of 1000 people within individual constituencies). Therefore, suggestions that supporters of Plaid are over-represented on the web can’t possibly be relevant to the apparent tendency of these Ashcroft polls to over-state Plaid support.

A second point is that, to my mind at least, the over-statement of Plaid support in these polls, and particularly on the constituency-specific vote, remains somewhat puzzling. It is not as if polls generally have tended to over-state Plaid support – either in the past or this year, where the eve-of-voting YouGov poll got, on its generic voting intention question, Plaid support more-or-less dead on. Moreover, if we look to Scotland, we find that both the Ashcroft constituency polls and the nation-wide ones were pretty good indicators of the major SNP advance. So quite why Ashcroft’s polls should have tended to over-state Plaid’s support, in seats where they were never in contention, remains something of a minor mystery within Welsh psephology – something for us to puzzle over during the long winter nights a few months hence.

Comments

  • J. Jones

    That’s quite an over-representation of Plaid supporters in the polling but Plaid have always been disproportionately visible on the web….perhaps in the pollster’s panel as well(?)

    • Roger Scully

      The Ashcroft constituency polls were all done by phone; even the online pollsters with the very largest panel (YouGov) wouldn’t have enough people to be able to do samples of 1000 per parliamentary constituency.

      • J.Jones

        So much for that theory then….do you know if they used staff from England or locally from Wales? Do you know if they offered a Welsh language option like Beaufort?

        • Roger Scully

          I believe that the fieldwork for pretty much all the Ashcroft constituency polls was done by Populus. As the Welsh ones were just part of a much bigger GB-wide project, I would be very surprised if they offered a Welsh-language option – I’ve certainly seen absolutely no indication of it anywhere in all the details I’ve looked at. I’m not sure which call centres Populus use – whether they have their own, or in turn sub-contract some or all of the work out.

  • Rowland

    While the constituency specific question was an interesting addition, it ultimately proved pointless, because the electorate were largely thinking about the Prime Minister, and which Prime Minister they would prefer. This ‘Preseidential’ type issue had been cleverly pushed by the Conservatives and their media friends for five years.
    This might answer the question regarding Plaid’s over representation in the polls. Leanne Wood was never going to be PM, neither did Plaid manage to convince the electoral that they were the best placed party to stand up for the few issues which concerned them (as opposed to Nicola Sturgeon and the SNP in Scotland) so when it came to the crunch the electorate reverted to the old duopoly.

Leave a Reply

Your email address will not be published. Required fields are marked *