One of the things we wanted to do in the voter study element of this year’s Welsh Election Study was to probe the political knowledge levels of voters (and, indeed, non-voters). Political knowledge levels are interesting in and of themselves – it’s important that we have some idea how much people know about the way in which they are governed etc. But knowledge levels can also be interesting to examine in terms of their relationship with other factors – such as vote choice, or attitudes to devolution.
So, we put quite a lot of time and effort into thinking about questions in our voter surveys that would probe the levels of relevant political knowledge possessed by our survey respondents. We explored political knowledge in several different types of ways. We asked people about their knowledge of which party (or parties) had been in government in Wales in the previous five years. We asked respondents questions about their knowledge of who held important political roles in Wales, the UK and the EU. And we also asked a series of questions which I’ll explore a bit further here, and in a following post, where we sought to investigate our respondents’ knowledge of their different elected representatives.
At present, above the level of councillor, each of us in Wales has ten of our own elected legislative representatives. Yes, that is right – ten:
- An MP for our parliamentary constituency
- The constituency AM for that same constituency
- The four regional AMs for the broader region in which one’s constituency is located
- The four MEPs who represent the whole of Wales in the European Parliament
We wanted to explore whether people had much, if any, idea of who these different representatives were – and whether there were significant or systematic variations in levels of awareness of the different types of representative. So we asked a number of questions about this. They were fielded in the pre-election wave of this year’s Welsh Election Study: surveys fielded in March, before the formal campaign for the Assembly election began. We deliberately fielded the questions then, before there was any impact of the campaign in raising the visibility of many of these politicians.
We were exploring these matters through a series of questions implemented on internet surveys. That immediately raises a potential problem: how do you stop people simply Googling the answers? Our solution – one that had already been tested fairly successfully by our partners at YouGov in various other surveys – is to give survey respondents a time limit on each question, to inhibit them from internet-searching the answers.
So we prefaced our knowledge questions about elected representatives with the following statement:
“On the next few pages we would like you to answer a number of questions about politicians. You will be given 30 seconds to answer each question and if you do not know the answer please just click the right arrow.”
We then asked the following series of questions:
“Which of the following people is the MP in your UK parliamentary constituency?”
“Which of the following people is the National Assembly for Wales member for your constituency?”
“In addition to your constituency member you also have four regional representatives for [NAME OF REGION] in the National Assembly. Which, if any, of the following people are among your regional representatives?”
“Wales is represented by four Members in the European Parliament. Which, if any, of the following people are among Wales’ four representatives in the European Parliament?”
As you can see, we asked each of our survey respondents about each of the four different types of elected representative that they have. (I should perhaps mention that we had checked the postcode of all respondents, to ensure that we were allocating them to the correct constituency and region). Underneath each question a respondent was presented with a list of names. For the MP question, the following names were offered to respondents:
- NAME OF ACTUAL MP
- Andrew Farmer
- Morgan Thomas
- Sarah Cole
- Rhiannon Griffiths
- James Evans
The order in which the names were presented to respondents was randomised – so, in other words, the ‘correct’ answer was not always the first one.
For the constituency AM question, the following list of options were given:
- NAME OF ACTUAL AM
- Gemma Davies
- Simon Jones
- Lowri Jenkins
- Paul Granger
- Martyn Hughes
Again, the order in which the names appeared on screen was randomised.
For the regional AMs and MEPs, things were a little more complex. We didn’t want to try the patience of our survey respondents too much – these questions were being implemented at the end of an already lengthy survey. Rather than probe them at tedious length about every single one of the four regional AMs and the four MEPs, we used the questions listed above and gave people the following options.
For regional AMs, people were given the following (again, randomised) list of names:
- NAME OF ONE OF ACTUAL REGIONAL AMS
- NAME OF A SECOND ACTUAL REGIONAL AM
- Robert Myers
- Mohammed Salim
- Sara Burns
- Carys Price
In short, people were offered a list that included two of their actual regional AMs, plus four other names. The order in which each of the four regional AMs was listed on the survey was also randomised.
A more-or-less identical format was then used for asking about MEPs. Respondents were offered the following list of names when asked about their European representatives:
- NAME OF ONE OF ACTUAL MEP
- NAME OF A SECOND ACTUAL MEP
- David Sherwood
- Elwyn Davies
- Lynn Goodwin
- Jenny Green
Again, which two MEPs each survey respondent saw listed was randomised, as was the order of the names listed.
There are a couple of further things to say about the information that we gathered from these questions. The first is simply to acknowledge that the question format was different between the two types of constituency representative on the one hand, and the regional AMs and MEPs on the other. I’m not sure it would have been possible to arrive at a better formulation that avoided any differences. But we should recognise that any differences in levels of public name recognition may be related, at least in part, to the question format. For the regional AMs and MEPs, some of our respondents may not have been expecting to see two correct names listed, and so may have moved on after finding one. That may reduce the overall levels of name recognition we see in the data.
A second, more general point is that these questions test name recognition, rather than name recall. Numerous surveys in various contexts have, in the past, simply asked people to name a particular elected representative, without giving them a list of possible names to choose from. Such tests of name recall appear to be much harder for many people than tests of name recognition – where the correct answer is given within a list. At least some people who would struggle to name a given politician when not given anything to prompt them do seem to have their memories jogged when they see the name written down. So the questions here are a relatively easy test of public awareness of their elected representatives. We may want to bear that in mind when considering the findings.
In my next blog post, I’ll present some of the findings from these questions. It is fair to say, I think, that not all of them are particularly encouraging…