One of the innovations seen in the period prior to the 2015 general election was the large number of constituency polls published – nearly all of them by Lord Ashcroft. We had quite often seen ‘marginals’ polls reported previously, but those were usually done with a standard sample size spread over a number of marginal seats – and thus having only a small number of respondents in each constituency. What was new about Lord Ashcroft’s polls was full-scale samples done in a large number of individual seats.
Another innovation of Lord Ashcroft’s polls was to ask, in all those conducted, two different voting intention questions. The first one was a standard, ‘generic’ question:
‘If there was a general election tomorrow, which party would you vote for?’
This was then followed by a question which specifically prompted respondents to think about the particular dynamics in their own seat:
‘Thinking specifically about your own parliamentary constituency at the next general election and the candidates who are likely to stand for election to Westminster there, which party’s candidate do you think you will vote for in your own constituency?’
The responses to this latter, constituency-specific question were generally used as the ‘headline’ figures reported from the polls. Now that the election is over, we can evaluate how these constituency polls, and the two voting intention questions contained within them, performed in relation to the actual election results across the five Welsh seats that Lord Ashcroft polled – Cardiff North, Cardiff Central, Brecon & Radnor, Carmarthen West & South Pembrokeshire, and the Vale of Glamorgan.
Before we do so, one word of caution should be entered. A poll is always an attempt to measure opinion at the time it was taken: in the phrase Lord Ashcroft himself likes to use, it is ‘a snapshot, not a prediction’. Some of the Ashcroft constituency polls were done quite some time before the election, and the further away from the election a poll is taken the less reliable we should expect it to be as a guide for how the election actually turns out. Please bear that in mind.
That qualifier entered, below I present five tables, for each of the five seats polled. (For Cardiff North, which was polled twice by Lord Ashcroft, we’ll take the more recent of the two polls). For each party the table will list three key pieces of information:
- That party’s share on the ‘generic’ voting intention question
- That party’s share on the constituency-specific voting intention question
- Their actual 2015 general election result in the seat
The tables also list, at the bottom of the ‘generic’ and ‘constituency-specific’ columns, the overall mean error: how much, on average, the polling figures differed from the actual election result per party.
Cardiff North (July 2014)
Cardiff Central (September 2014)
Brecon & Radnor (November 2014)
Carmarthen West & South Pembs (Dec 2014)
Vale of Glamorgan (February 2015)
So what can we make of all that? A few observations:
- When comparing the elections outcomes to the results of the two voting intention questions, the constituency-specific question was, overall, closer to the result in three of the seats (albeit only very narrowly, by 0.16% in one instance) while the generic question was closer in the other two. In four out of the five seats (all except Cardiff North) the generic question ‘predicted’ the correct winner; the constituency-specific question got it right in three out of five (incorrectly ‘predicting’ the Liberal Democrats to win in Brecon & Radnor). Certainly in Wales there was no demonstrable superiority to either form of question.
- When we look at specific parties, we do find that the constituency-specific question was closer to the correct result for the Liberal Democrats in the two examples where they were defending their own seats. But for Plaid Cymru, the generic question was closer to the final result in all five constituencies.
- There was also no very obvious trend for the Ashcroft polls to be closer to the actual result the closer they were conducted to the election.
Overall, were these constituency polls worth it? Perhaps that depends on whether you have a few tens of millions of pounds going spare… We await a full analysis of all the Ashcroft constituency polls across Britain. But the experience of them in Wales does suggest that they were certainly not exempt from some of the problems that faced the opinion polls in 2015.
Postscript (02/06/15): Just a brief addition to this post, in response to some comments both here and on Twitter, about the performance of Plaid Cymru in these polls. The first point is a reminder that all the Ashcroft constituency polls were conducted by telephone. (There is a very good reason for this: even the company with the largest on-line panel, YouGov, would struggle to find samples of 1000 people within individual constituencies). Therefore, suggestions that supporters of Plaid are over-represented on the web can’t possibly be relevant to the apparent tendency of these Ashcroft polls to over-state Plaid support.
A second point is that, to my mind at least, the over-statement of Plaid support in these polls, and particularly on the constituency-specific vote, remains somewhat puzzling. It is not as if polls generally have tended to over-state Plaid support – either in the past or this year, where the eve-of-voting YouGov poll got, on its generic voting intention question, Plaid support more-or-less dead on. Moreover, if we look to Scotland, we find that both the Ashcroft constituency polls and the nation-wide ones were pretty good indicators of the major SNP advance. So quite why Ashcroft’s polls should have tended to over-state Plaid’s support, in seats where they were never in contention, remains something of a minor mystery within Welsh psephology – something for us to puzzle over during the long winter nights a few months hence.