Be A Sceptical Economist

Small Furry Animals

It’s unsurprising that behavioural finance is gaining ground in economic circles, as the wanton failures of more traditional versions have become all too clear over the last decade. However, just because psychology can provide us with valuable insights into the workings of finance there’s no need for us to drop our defences. Like any new subject there’s plenty to be sceptical of.

One of the biggest problems is one that psychology has faced since it began, to do with the authenticity of its experiments. If you take a bunch of people, shut them in a darkened room and make them do odd things, out of context, you might expect them to behave a bit peculiarly. Psychologists, on the other hand, tend to live in darkened rooms and regard doing odd things in them, often involving small, furry animals, as quite normal. So they see nothing wrong in this and are quite happy to write up the results of these experiments in order to help deepen our knowledge of human behaviour.

Bias in the Laboratory

Unfortunately, in general, what they’ve actually been doing is deepening our knowledge of what people do when you shut them in darkened rooms and make them do odd things, out of context. As the links between psychology and economics grow ever tighter it’s becoming obvious that similar strictures apply to the latter topic. This lack of experimental authenticity has been a long term issue for the behavioural economist John A. List and some of his recent work with Steven Levitt helps to shine a baleful light on the tricky business of experimenting with human beings.

In fact psychologists have known for decades that the experimental situation can bias results. There’s been a full-blown debate about this which has largely ended with the people in opposing corners making rude hand signals at each other and going “tra-la-la” while holding their hands over their ears. Underlying this is an argument about whether you can experiment on people in the same way you experiment in the physical sciences.

Hard Science, Soft People

If you model psychological research on the hard sciences you really need the control that laboratory set-pieces allow. However, as opponents of this approach have continued to point out, you can’t treat a human being like atoms or molecules or other inorganic stuff. People can’t be relied on to do exactly the same thing every time (neither can atoms or molecules, but that’s a story for another day) because they learn from experiences, change their behaviour and sometimes just get up in a grumpy mood. So even within the controlled environment of the lab there are major problems with experimenting on people.

Well, actually that’s not quite true. The experiments are easy enough, it’s drawing conclusions that are difficult. In theory, in the physical sciences if you design a careful experiment then someone following your instructions should be able to repeat it and get the same results (in practice this also isn’t quite true). In the social sciences you can roughly expect that most of the time when someone else repeats the experiment they’ll get a different result. Sometimes a completely different one.

Economics is Money and Morals

In What Do Laboratory Experiments Tell Us About the Real World? List and Levitt have listed a whole range of reasons why this might happen and, in particular, draw attention to an important facet of economics. Although they ascribe to the idea that humans act in their own self-interest to maximise what economists call “utility” they point out that you can’t equate utility entirely with wealth. As we discussed in Adam Smith’s Monkey Business, maximising utility is a combination of wealth and morals. So, for example, in some cultures you’ll usually tip a waiter in a restaurant but you’ll tend to do so more if you’re on a first date. If you change the social context you’ll change the economic response.

So, in lab experiments, the very fact that you know you’re being observed can put you on your best behaviour – for instance, participants contribute far more to charitable causes in lab conditions than in real-life. List and Levitt suggest that, on one hand, making sure that the experiment is anonymised can help with this but then wryly point out that participants may detect this and modify their behaviour because they’ll figure out that the researchers think that anonymity’s important. Humans are the original riddle, all wrapped up in an enigma.

List Me The Ways To Go Wrong

The list of factors that may invalidate psychological experiments is long and wearisome, but the writers do a good job of describing them. Observation and social context are major issues. So is the problem that lab experiments tend to self-select: the average lazy bum can’t be assed to drag themselves out of bed to solve complex and meaningless puzzles for researchers. If the average lazy bum is a typical human then this will skew the results. Ditto the fact that participants are drawn predominantly from educated groups.

Finally you have the problems of observer bias, where researchers only see what they want to. This is a problem that’s far from exclusive to the social sciences. For instance, palaeontologists long failed to spot and accept the similarities between human and ape physiology, despite it being obvious in retrospect. Social conditioning, which in this case meant denying a simian ancestory regardless of evidence because religious teachings said it wasn't so, can heavily bias researchers who see only what they want to surprisingly often, given the supposedly objective nature of science.

Despite this, the laboratory experiment remains the most popular way of doing psychology largely because it’s the only one where you have any chance of controlling what’s going on. And, of course, it’s easier. The alternatives are to find natural data sets to analyse or to create so-called field experiments where the researcher creates an experiment outside of the laboratory, usually without people knowing what’s going on.

Field Experiments and Natural Settings

John List has a distinguished record of creating field experiments, some of which we’ve discussed in the past while Steven Levitt is famous, outside the world of economics, for Freakonomics, a book that largely describes the outcome of analysing data generated in natural settings. Although these situations have significant advantages over the laboratory because they’re far more unlikely to cause people to change their behaviour simply because of an artificial social situation, they also have the downside of being difficult to set up and tricky to control.

Control is the key issue here, because the laboratory setting means that it’s possible to manage for what are known as confounding variables. So, for instance, heavy drinkers die young – therefore presumably drinking heavily is the cause of early death? Well, maybe, but other people from the same social groups who don’t drink heavily die don’t live any longer – so maybe it’s social status or background that’s the real cause of early death. In the real world you can’t control for this type of confounding variable.

The Confounding of the Hawthorne Effect

The classic example of this is the Hawthorne Effect. This arises out of series of experiments at the Hawthorne Plant in the 1920’s where the effects of changing levels of illumination on productivity were investigated. What the experiment purported to show was that it was the effect of studying people, and the people knowing this, which altered their behaviour rather than the change in lighting. This, of course, is the problem of knowing you’re being observed re-asserting itself, but in a natural environment.

Yet, when List and Levitt investigated this effect they ended up concluding that the “evidence for a Hawthorne effect in the studies that gave the phenomenon its name is far more subtle than has been previously acknowledged”. Basically, the apparent improvement in productivity due to observation effects was much, much less than previously reported. All of which indicates that the problems of lab experiments don’t simply go away when you step out of the door into the real world: the authors review the problems, but also the possilities, of field experiments in Field Experiments in Economics: The Past, The Present and The Future.

Be Sceptical

None of this means that we should give up. Knowledge is built slowly and uncertainly, with many false steps and the best evidence comes from multiple sources – from the lab, the field and natural data from multiple cultures. Some of the current findings of behavioural finance will doubtless prove illusory but overall the application of psychology to economics promises a rich reward for both subjects.

However, that doesn’t mean we should stop being curious and simply accept what we’re told. As ever read everything – including The Psy-Fi Blog – with a sceptical mind.


Related Articles: The Death of homo economicus, Adam Smith's Monkey Business, Moral Corporations: An Oyxmoron?