(This blog-post, and two following ones, have been prepared with the very considerable assistance of Matt Congreve, a Cardiff University student currently working for the Wales Governance Centre as part of the Cardiff University Research Opportunities Programme. You can follow Matt on Twitter: @CongreveMatt)
One of the key priorities of the Voter Study component of this year’s Welsh Election Study was to explore what political issues were most important to people in Wales in 2016. We wanted to find out what things really mattered to people.
This is something that some Welsh Political Barometer polls have looked at before (see, for example here). However, there was a crucial difference between what past Barometer polls have done and how we explored voters’ issue priorities through the Welsh Election Study.
When we examined these matters in the February Barometer poll, we asked the following questions:
“Which of the following do you think are the most important issues facing the country at this time? Please tick up to three.”
“And which two or three of the following will be the most important issues for you in the upcoming Welsh Assembly election?”
Respondents to the Barometer poll were then given a fairly long list of possible categories from which to choose. Obviously we chose the options given to respondents with some care. (And we also gave them an ‘Other’ option, which if chosen allowed them to type in their own response). Nonetheless, this question format still depended on us defining the issues for people, and then having them choose.
However, it is always worth bearing in mind that, as the late Philip Gould put it, “people do not think in predictable ways or conform to conventional prejudice” (The Unfinished Revolution, p.328). There is always the risk that a question with pre-prepared answers is steering people to issues that we, the researchers, think of as the important ones – rather than those that people are actually most concerned about.
In the Welsh Election Study, therefore, we followed the lead of several other academic election studies in the past (such as the British Election Study). We did this by offering our respondents questions that sought to get our respondents to tell us what issues they thought mattered, without any prompting from us.
We did this through asking three ‘open-ended’ questions in the Welsh Election Study. ‘Open-ended’ simply means that no pre-defined answers were offered to respondents; the contrast is with ‘closed-ended’ questions, where respondents are invited to choose between a set of pre-defined answers.
In the pre-election wave of the Welsh Election Study (conducted in March, before the official campaign had begun), we asked our respondents two questions:
First, we asked a standard question used in several past British Election Study surveys:
“As far as you’re concerned, what is the single most important issue facing Britain at the present time?”
We then followed this up with:
“As far as you’re concerned, what is the single most important issue facing Wales at the present time?”
For both questions, respondents were simply offered a text-box into which they might type an answer. They were also asked, immediately after each question, which political party they thought was best able to handle the issue that they had listed as the most important one.
Later on in the Welsh Election Study – in our post-election survey, which was conducted immediately after the election was over – we also asked the following question of our respondents:
“What was the single most important issue for you when deciding how to vote in the Welsh Assembly election? (Please type your answer in the box below. If you think there were no important issues, please write NONE)”
The great advantage of these open-ended questions is that they allow people to respond in the most unmediated, un-prompted manner possible: to tell us what they think matters.
The great disadvantage of these questions is that they take a lot of work to analyse. (That is why we didn’t use them for our Barometer poll – it would have taken far too long to code the results.) You need to work through the answers to each of the questions given by every single survey respondent. (And we had over 3,000 respondents for each of the Welsh Election Study waves). You need to allow for the various mistakes that people make. (I never knew that there were quite so many ways to spell ‘immigration’). You then have to code the responses manually into categories. We don’t yet have machine-coding that is good enough to deal with these responses, while the categories into which the responses should be coded are not self-evident; they require a great deal of thought and interpretation.
However, with the considerable assistance of Matt (who surely deserves to have the Order of D’Hondt conferred on him), we now have fully-coded responses to our open-ended questions in this year’s Welsh Election Study. Over the next three blog posts I’ll set out the results – they are interesting and, in one or two places, a little surprising.