Polling

New RMG Poll for the IWA

The Institute of Welsh Affairs has today published results from a short poll that it has run with the survey firm RMG. I was consulted about the wording of a few of the questions, and below is the text of an article that I wrote for the IWA Blog, Click on Wales, giving my assessment of the results. (EDIT, 07.47, 26/11/13: Further details of the poll are here)

Today’s RMG poll for the IWA offers us some useful information about current public attitudes to devolution in Wales. Though the sample size of this particular poll is a little less than ideal, it nonetheless provides some valuable insights into a number of important issues facing Wales today.

The poll covers four matters. The first concerns public awareness of recent devolution-related events. Awareness is a difficult issue to get at via relatively short polls: uncovering true levels of knowledge is tricky to achieve without asking questions that risk making survey respondents feel stupid or insulted. Alternatively, asking people about how well informed they think they are can fall victim either to people deluding themselves about their level of knowledge or to them offering ‘socially desirable’ but quite untrue responses. To try to combat this (and in line with an approach commonly used in surveys) the question used here sought to reassure respondents that it was OK to acknowledge ignorance:

“Some people have told us that they did not know about the announcements made recently about the extended powers being given to the Welsh Assembly. Could I ask you whether you are aware of these announcements?”

Despite the clear slant of the question, 74% of respondents said that they were aware of the announcements of extended powers, while 26% were not aware of the announcements. This is encouraging – three-quarters of people seemed to know at least something about what had been announced – although there was not space in the poll to explore further the depth of people’s knowledge. The general pattern observed in major surveys conducted in recent years – such as the 2011 Welsh Referendum and Welsh Election Studies – is that most people in Wales seem to have a decent grasp of the basics of devolution. They may have some more detailed knowledge in areas that touch directly upon their lives, but other than that the subtleties tend to elude most of them.

The second matter covered by the poll was the element of the recent UK government announcement concerning borrowing powers for the Welsh Government.

It has been proposed that the Welsh Government will be able to borrow money to pay for major projects such as upgrading the M4 motorway and similar types of projects that could help the economy of Wales. Do you think this will benefit the Welsh Economy?”

Given the highly negative tone of political debate and media coverage in recent years about deficits and government borrowing, I had expected that there would be substantial opposition to the idea of allowing the Welsh Government to borrow. However, the results of the poll showed a clear majority – 59% of respondents – believed that these proposed powers would benefit the Welsh Economy, compared to 25% of respondents who felt they would not, and 17% of respondents who felt that these proposals would not make much difference. The subject needs to be explored further, of course, but these findings do suggest broad public support for borrowing powers.

The third topic covered by the poll was the thorny question of income tax powers. The question posed on this tried to pitch the subject in as neutral and unbiased a manner possible:

“Would you be in favour or against the Welsh Government having the power to raise or lower income tax rates in Wales?

Note that this question is not asking about voting intentions in a referendum on income tax powers. On the question asked here, 40% of respondents replied that they would be in favour, 43% replied that they would be against, and 18% replied that they did not know. This compared with a question specifically asking about referendum voting intention, asked by YouGov for ITV-Wales in February, where the balance of opinion was 39% to 34% in favour (with the remainder undecided); and the Silk Commission’s research by ICM in May 2012 which – on a slightly different question again – found a 64% to 33% balance in favour of some income tax powers.

Some might see these results as suggesting a trend away from support for income tax powers. I think it would be unwise to conclude that – at least not yet. It is probably more likely that public attitudes are still rather inchoate, and thus highly sensitive to minor variations in question wording. We need more regular polling on income tax powers before we can say that a clear balance of opinion in the public on this issue has been established.

Finally, the poll touched on more general attitudes towards devolution. Here, and despite the apparent caution over income tax, the results were very much in line with what all major surveys over recent years have shown. There is very little support in Wales right now either for independence or for abolishing devolution. The overwhelming majority of people in Wales now support devolution. The major divide is between those who are reasonably happy with the level of autonomy that Wales currently enjoys and those who wish things to go further:

·        12% of respondents were of the view that there should be no devolved Government in Wales

·        5% of respondents were of the view that the National Assembly for Wales should have fewer powers

·        37% of respondents were of the view that the Assembly’s powers should be left as they are

·        40% of respondents were of the view that the National Assembly for Wales should have more powers

·        7% of respondents were of the view that  Wales should become independent

 

In summary, this poll reinforces the consistent picture from evidence over recent years that devolution is now the settled will of the Welsh people. But while the fact of devolution now has stable majority support, the extent of it does not. And whether partial autonomy for Wales should include tax powers is something about which the people of Wales, as well as their political leaders, remain divided.

Comments

  • Albert Jones

    I don’t think the wording of the income tax question fails on neutrality, but it does seem a bit open to interpretation.
    Do you think something like the following would offer a clearer all-round picture?

    Which government or governments should, in your view, be responsible for setting income tax rates in Wales – the UK government, the Welsh Government, or a combination of the two?

  • Michael Haggett @ Syniadau

    Some more details of the poll are now on the RMGClarity blog, here.

    There are some graphics, the size of the sample was only 500, and its unweighted. But it’s useful information, and I’m glad the questions were asked.

  • J Jones

    Have you any comment on the drop in the percentage in favour of devolution of income tax varying powers from the 64% in favour in the ICM poll for the Silk Commission to the 40% in favour in this poll? Quite a change. It’s mirrored by a shift from 9% in favour of abolition of the Assembly to 12%. A small shift admittedly but at extreme ends of the scale you would expect little variation.

    • Roger Scully

      If they were the same question in the two surveys it would be a big change. But they are worded differently, and I would suspect that it is simply a question wording effect. That’s why we need more regular polling on some of these issues, to be able to trace any genuine changes, and distinguish them from merely differences in the questions asked.

      On the other point – a 9% to 12% shift is hardly striking. The Beaufort poll figure was particularly low – maybe due to their sample generally being a bit more pro-devo than most samples, as you have suggested. The 12% figure here is closer to what we have generally found over the last few years. Even if public attitudes were wholly unchanged, you would expect sampling variations and other essentially random factors to produce changes some changes from poll to poll – all the more so given the relatively small size of the sample here. If we start to see a series of polls showing a steadily increasing proportion of the population supporting a No Devolution option then that would be very interesting. But there’s no real sign of that yet, I don’t think – nor has there been for a decade or so.

      • J Jones

        So:-

        Q3. Would you be in favour or against the Welsh Government having the power to raise or lower income tax rates in Wales? 40% say YES.
        Q6 Generally speaking, do you think that the Welsh Government should or should not have the power to decide the level of income tax in Wales? 64% say YES they should.
        But also ICM:-
        Q4 And which level of government do you think SHOULD have the most influence over the taxes you pay? 35% say the Welsh Government.

        OK, I get it now.

        • Roger Scully

          Yes – what you generally find is that as opinions on an issue get more clearly defined, the wording of the question makes rather less difference. Asking about whether Northern Ireland should stay within the UK, for example, might be an instance where you probably wouldn’t expect large question wording effects.

          Income tax devolution in Wales, however, is an issue where I suspect most people have a rather less developed attitude. That may change somewhat if the issue becomes more widely debated. But for now it’s the sort of issue on which you would expect reasonably substantial wording effects.

  • J Jones

    As you know Roger I have reservations about the Silk Commissions polling. A major failing is that both the ICM poll and the later Beaufort poll were too ambitious. They tried to get all sorts of opinions and nuances of opinion when they should have used more factual questions to establish the nature of their sample and restricted the opinion questions to brief, easily understood concepts.
    It is better to ask a respondent; “Where were you born?” than to ask a series of questions about their feeling of national identity: Would you consider yourself:-
    Welsh?
    Welsh and British?
    More Welsh than British?
    More British than Welsh?
    British?
    Other?

    A major problem is the number of times that “Welsh” is said and the juxtaposition of Welsh and/or British. What the pollster is doing is establishing a scenario where the respondent is pushed towards an identity that he probably hasn’t considered in a lifetime. But having established a Welsh identity the respondent is more likely to go along with the suggestion that…..he speaks Welsh, naturally he feels pride in Wales and therefore thinks that Wales should control its own affairs etc etc.

    Factual questions like: In the 2011 census did you say that you were a Welsh speaker? are far more useful in weighting the profile of responders than “Do you speak Welsh”.

    If Silk had really wanted to know how people thought a 2000 person survey should have been limited to just a very few opinion questions……much like the RMGClarity poll in fact.

    • Albert Jones

      The question you should ask yourself is – just because you see the world in a particular way, why should you take for granted that others around you see things similarly?
      However, in terms of the research, it appears you’re claiming that simply asking about the existence of Welshness is at least partially responsible for promoting Welshness as an identity.
      Couldn’t those of the opposite view claim exactly the same thing about Britishness?
      Your view seems connected to a suggestion in your comment that Welshness is a recently constructed thing – although history appears relatively conclusive in contradicting you in that.
      Maybe a test of your theory would be to offer a third option in such polling – a purely made up identity of non-descript nature which could be offered alongside Welsh and British. Over time we could see how this new identity gathers momentum, if at all.
      I guess if you’re correct, in a few decades we’ll be facing calls for the creation of a new state by the non-descript-made-up-identity-ists.
      I’m sure experienced pollsters will be eager to try this out!!

  • J Jones

    I don’t know exactly where you are going with your comment Albert but I didn’t pluck the theory out of thin air. The idea that asking Welsh Identity questions had an influence over the answer to subsequent questions was explored by the ONS after the 2001 census. In particular they wanted to look at why they had a higher percentage responding to the question “Can you speak Welsh?” in the Welsh Local Labour force survey than appeared in the 2001 Census. This is a quote from that research paper:-

    “The answers a respondent gives to one question may be affected by the context it is
    asked in. The context will vary according to the main subject matter of the survey,
    the social and political climate of the day, the way the survey was introduced as well
    as the answers respondents have given to previous questions.
    On these surveys, the context of the Welsh language questions varied. Not only did
    they cover very different subject matter but some surveys included questions about
    Welsh identity while others did not.”

    It appears that asking “do you consider yourself to be Welsh” Then asking “Do you speak Welsh?” boosted the numbers claiming to speak Welsh. The ICM survey for Silk asked Identity questions, questions about Welsh language speaking and opinions about devolution of taxes….particularly in this instance devolution of Income tax varying powers.
    What we know about the 2000 people who answerd the questionnaire is that 60% claimed to be able to speak Welsh. This should be compared with the 16% of adults in the 2011 census who claimed to be able to speak Welsh.

    You can either take the view that this sample was not representative of the population or you can take the view that the questionnaire was structured in such a way as to induce people to claim a greater degree of Welsh “credibility” than was reasonably true.
    My belief is that both things are true to a degree; the sample wasn’t carefully weighted to 16% Welsh speaking and 31% not born in Wales and the questions influenced responses by context.

    The result is that Silk produced polls that themselves influenced opinion in the people responding and then were further used to re-inforce the outcome desired by the Commission. It’s only when you compare one poll with another that you have to consider the validity of a 24% difference in opinion.

    Roger is suggesting that the two questions were slightly different and this explains the 24% change but look at the wording of the two questions:-

    Q3. Would you be in favour or against the Welsh Government having the power to raise or lower income tax rates in Wales?
    Q6 Generally speaking, do you think that the Welsh Government should or should not have the power to decide the level of income tax in Wales?

    Which question makes you feel more inclined to answer “YES” to more income tax powers?
    For myself, I feel no particular urge to answer either question differently and this is why I suggest that Context is the key to the huge difference.

    • Albert Jones

      Roger’s point re the amplified effects of even slight variations in questions is undoubtedly very significant with this issue.
      However, the suggestion that this has somehow been harnessed by an independent and cross party commission in order to skew polling results/conclusions is quite an accusation.
      I’m not sure who is meant to be driving this conspiracy, but I wish you every success in exposing them.

  • J Jones

    I don’t know whether they are knaves or fools Albert but, looking at the members of the Commission, I don’t see much history of devosceticism. In reality their surveys were just what I have said; very badly designed. I would have commented on their own open discussion forum but, guess what, they blocked every post that I made pointing out the short-comings of their surveys. So much for a transparent and open discussion of the question of further devolution.

Leave a Reply

Your email address will not be published. Required fields are marked *