Well, that’s one way of looking at it – but probably not the most sensible way.
The annual BBC/ICM St David’s Day poll has been published. (The full results are now available here). There are a number of interesting findings in the poll, many of which were discussed on BBC Wales yesterday. I’ll be reviewing some of them in more detail in later posts on the blog.
But I thought I should turn first to one particular result. Many of you will recall that we have had some debate and discussion here previously about the questions on ‘constitutional preferences’ that the BBC/ICM polls, and other ones, have generally run. In particular, I discussed the changing wording of the questions here. I also (somewhat belatedly) discussed here the results of the last such question run on a BBC/ICM poll. That previous poll had been conducted in the immediate aftermath of the Scottish referendum, and produced two particularly notable findings:
- Support for ‘More Powers’ for the Assembly was an unusually high level, at 49%: the highest level in any such BBC/ICM poll, and one of the highest levels on any even vaguely comparable question in any poll in recent years;
- Support for Independence was, at 3%, just about the lowest ever found in any such multi-option constitutional preference question.
It was the latter finding that much of the media, and many politicians, seemed to seize upon, and which excited a great deal of comment. As I suggested in my review of that poll and how its results had been reported, I thought a lot of that discussion was greatly overdone. It ignored a number of obviously pertinent considerations, most particularly:
- That such polls have a 3% ‘margin of error’. The 3% choosing the independence option was within the margin of error of the finding of the previous such survey, which had found 5% selecting the independence option. So maybe all the September poll was showing was normal sample variation amidst a picture of no real change.
- There was a very obvious contextual factor that might explain why this particular poll could have produced findings a little out of kilter with the norm. It was conducted immediately after the Scottish referendum result, where independence had been rejected but there had been considerable discussion in the final few weeks of ‘more powers’ for the Scottish Parliament. In such a context, finding a low level of support for independence and a high level of support for more powers was not merely unsurprising; it was eminently predictable.
So now, more than five months on, the same survey company (ICM) have asked the exact same question to a new set of respondents in Wales. What did they find? The table below presents the results, and the percentage changes from the September 2014 poll:
“Which of these statements comes closest to your view?”
|Wales should become independent, separate from the UK||6% (+3)|
|The Welsh Assembly should have more powers than it currently has||40% (-9)|
|The powers the Welsh Assembly currently has are sufficient and should remain as it is now||33% (+7)|
|The Welsh Assembly should have fewer powers than it currently has||4% (+2)|
|The Welsh Assembly should be abolished and Wales governed directly from Westminster||13% (+1)|
|None of these||1% (-2)|
|Don’t Know||3% (-)|
So, what can we make of these figures, and the changes since the September poll? Well, I suppose I could also have headlined this piece ‘Support for Assembly to have fewer powers doubles”! But although strictly accurate, that would have made no more sense than to focus on the apparent doubling of support for independence.
What I think we are seeing here is simply a reversion to a much more normal pattern of results. The findings of this new poll are much more typical of what the BBC/ICM polls, and other similar ones, have been finding pretty consistently in recent years. We see support for independence, on this sort of question wording, a little below 10%. We also see support for abolition of the Assembly, on this sort of question wording, at around about, or just below, 15%. And we see a very clear majority of respondents supporting either the status quo or a more powers option – with there normally being a modest margin in favour of the latter. This is very much the pattern that has generally been found in most studies across the last decade or so, and we have little evidence that it has changed very much over that period.
At least some of the readers of the blog will be familiar with Twyman’s Law, semi-seriously coined by Prof Michael Twyman, which states that “Any piece of data or evidence that looks interesting or unusual is probably wrong”. Prof Twyman’s injunction is very often relevant to the analysis of polls, and almost certainly was relevant to the constitutional preference findings of the September BBC/ICM poll. That is not to criticise the conduct of the poll – as I have said here before, ICM are a company with a deservedly high reputation. But they were asking about constitutional preferences in Wales at a particularly extraordinary moment in political life in the UK. Now that the dust has settled on last September’s referendum, the pattern of answers to this question about how Wales should be governed seems also to have settled down. The moral of the tale, I think, is clear: avoid getting too excited about the findings of any single poll, particularly one carried out at an unusual time.