On Monday 28 April Rob Evans and Martin Weinel were in Helsinki at the Experimental Social Science workshop organised by Professor Petri Ylikoski of Helsinki University. The workshop explored the different ways in which experimental approaches are being used within the social sciences.
This diversity was nicely illustrated by the three presentations that made up the morning session. All three reported on empiricial research and all three described quite different projects. Topi Miettinen described the use of experiments in behavioural economics to explore the relationship between social values and economic decision-making; Martin Weinel outlined the Imitation Game method and some of the main findings from our current project; and Anssi Peräkylä reported on two published studies in which conversation analysis was combined with physiological measurements in order to examine the relationship between the affective and linguistic elements of story telling.
In contrast, the afternoon session examined the idea of experiments and their role in the social sciences. Rob Evans described some of the wide range of research designs that could be seen as ‘experimental’ and argued for an fairly broad understanding of experiment, with different approaches linked by family resemblence rather than seeking to define a single research design as the ‘correct’ way to do experimental social science. Thus, for example, experiments can include those that aim to demonstrate a principle (e.g. the pitch drop experiment), those that aim to measure or detect something (e.g. LIGO), those that aim to test a hypothesis by making some kind of intervention in a very controlled setting (e.g. psychological experiments), those that aim to demonstrate a principle in a very applied setting (e.g. experimental archaeology) and those that occur when different organisations or institutions enact differnet polcies (e.g. natural experiments created by devolved governments in the UK). Seen this way, an experiment is not a single thing but the outcome of a set of decisions about:
- the importance of an experimental ‘treatment’ or ‘intervention’
- the importance of ‘deception’ in concealing the purpose of the experiment from participants
- the tension between creating an idealised artificial setting and using a more realistic field setting
- the importance of using a control or comparison group
- the way in which internal and external validity are managed in order to allow results to generalise from sample to population
In his talk Petri Ylikoski took a similar tack. Focussing on the rise of evidence based policy and practice, he argued that the evidence hierarchy that typically characterises such approaches may be counter-productive in the long-run. His argument was that setting one particular research design — typically the randomised control trial — as the gold standard for social science research may prove counter-productive in the long-run as the effect is to devalue other forms of research design and evidence. In other words, by taking too narrow a view of what counts as evidence, the evidence-based policy movement may find itself working with less evidence — and less reliable evidence at that — than would otherwise be the case.