Hindsight Bias

Behavioural Biases (2): Hindsight Bias

The CIA reports that hindsight biases are:

attributable to the nature of human mental processes, not just to self-interest and lack of objectivity, and that they are, therefore, exceedingly difficult to overcome.

And, after some years searching for weapons of mass destruction in the wrong country, as the North Koreans have demonstrated by actually exploding nuclear weapons, the CIA really ought to know about this.

In fact, if you replace “exceedingly difficult” with “impossible” you’ve basically got the point – hindsight bias, the tendency to believe that events that have already occurred were more predictable than they were before they took place, is endemic and a built-in part of the human psyche. It’s one of a multitude of factors that leads to investor overconfidence and underperformance.

Fighting Butterflies

Most investors can do something about most biases they face if they’re motivated to do so. We don’t need to succumb to overconfidence, for instance, and indeed we can guard against it. Unfortunately, as the CIA suggests, hindsight bias is seemingly immune to all attempts to protect against it. It's more than just a disposition towards wishful thinking, it's something to do with the fundamental wiring of the brain.

Studies of hindsight bias have been carried out in all sorts of spheres of activity – from intelligence gathering to politics and from medicine to civil engineering – and all find the same thing. When it comes to analysing our mistakes we can’t shake off the prism of what has already happened.

For investors, who as much as anyone, operate in a world full of frantically flapping quantum butterflies, this is a serious matter. If we can’t effectively review our past decisions intelligently because what actually happened keeps getting in the way we’re likely to experience overconfidence and regret. Overconfidence, because we don’t recognise that our past performance isn’t as good as it should be and regret because we think we should have done better than we have.

Confused? Welcome to the world of the average investor. Welcome to my world.

Retrospective Analysis

If you spend any time reading financial punditry you’ll probably be struck by the omniscience of the genre. Looking backwards the mistakes that have been made are obvious but we don’t live our lives in rewind. We have to do the best we can looking forward, not back, and so the opining of the modern day soothsayers isn’t of much use to us. For the most part none of them predicted the worst financial crisis in four generations, so expecting them to get anything less important correct is hopeless.

However, the constant reworking of the past through the filter of the present simply adds the problems we face in recognising the problems of hindsight bias. We operate as though it doesn’t exist and this collective blindness is hard to understand when the impossibility of the predicting the future from where we stand today is taken into account.

Investing in Hindsight

Goetzmann and Peles (1997) conducted some research on investor recollection of their returns against actual performance. For reasons that escape me their first group of subjects were architects, a profession not especially known for its investment expertise – although perhaps that was the point. Anyway, the research showed that in between designing carbuncles and flat-pack housing they recalled a performance that was over 6% higher than it actually was. Looking at people who thought they were actual investors produced a slightly better result – they only overestimated by over 5%.

(As a warning about the validity of data quoted in random blogs which support your intuitions - the numbers above were based on a microscopic number of responses. If anyone's really interested the research of Baruch Fischoff is a good start and bit of an eye-opener).

One possible explanation for this is the availability heuristic – the psychological trait by which we’ll tend to latch onto information that’s more readily available in order to answer a question. In effect something that actually did happen is more easily accessible in memory than something that didn’t. Looking back it’s hard to imagine alternative outcomes, even when those were what we expected at the time.

Another explanation is to do with cognitive dissonance – the process whereby people manage to believe two impossible things at the same time in order to avoid facing unpleasant realities. Like, presumably, admitting to the spouse that you’ve lost your pension pot buying small capitalisation mining stocks. There’s a strong psychological pressure to not admit to losses.

Remembering or Not

Most of what we read about the past was itself written after the event, it’s very rare that we can get contemporaneous reports of what people actually felt. However, Nicolas Taleb, in The Black Swan, provides a very neat piece of analysis and research to make the point. Drawing on William Shirer's on-the-spot diary of events leading up to the Second World War he shows how little understanding there was ahead of time of the effect of the forthcoming events.

Moreover it's clear that investors don't always appreciate the times they live in. Niall Ferguson's (deep breath) Political Risk and the international bond market between the 1848 revolution and the outbreak of the First World War shows in some detail that prior to the War to End All Wars most investors seemed to be expecting a relaxing holiday. Instead of which they sent their sons into the hands of Haig, who refused to visit military hospitals to avoid the evidence of his stupidity.

Still, if the world couldn’t see either World War coming what chance does the average investor have of predicting anything? Yet, we all have beliefs and we all invest on the basis of those beliefs. And then we rewrite those beliefs in our memories and fail to remember why we did what we did.

The Recreation of Identity

I suspect that there’s a pervasive effect going on, to do with the constant recreation of identity. Anyone who keeps a diary for any length of time will experience the odd feeling of reading old entries and discovering that they were written by another person, who seems to have views that you don’t hold anymore. Research into memory shows that we don’t simply recall memories from some database in the brain: we actively recreate them. New information and experiences can interfere with these recreations and produce apparently real yet actually false recollections.

The CIA’s report on the problem summarised it thus:

Unfortunately, once an event has occurred, it is impossible to erase from our mind the knowledge of that event and reconstruct what our thought processes would have been at an earlier point in time. In reconstructing the past, there is a tendency toward determinism, toward thinking that what occurred was inevitable under the circumstances and therefore predictable.

The suspicion is that this constant rewriting of memory is part and parcel of identity. We see our past experiences as a single, stream of consciousness but, in reality, it’s nothing of the sort. Our lives are fragmented by cognitive dissonance, we hold different and opposing things to be true at different times in our lives and we genuinely see no problem in doing so, because we don’t even recognise that we’re doing it.

Eradication is Futile

As the CIA pointed out, hindsight bias is virtually impossible to eradicate. It’s not something you can build defences against, learn new tricks and find ways of overcoming – it seems to be something absolutely fundamental to the whole human experience. Yet it can cause investor overconfidence in a very nasty way – if an investor continues to believe that they’re doing OK when they’re not then they won’t adjust their behaviour to improve. Regret may cause people to stop investing completely – a worse outcome than buying mutual funds.

What can we do about this? Well, there’s a very simple trick we can use. Keep a diary.

If, when we make an investment decision, we record the basis numbers and thought processes we can later revisit this to try and reanalyse what we did in the light of new information. This is a difficult thing to do, but learning to keep an open mind is important – if we continually revisit our investment decisions and force ourselves to face the information we had at the time of the decision, rather than the information we have now, we can begin to understand the issues at hand. We may even learn from the experience, although don’t count on it.


Previous Article: Behavioural Biases (1): Overconfidence and Over Optimism

Next Article: Behavioural Biases (3): Loss Aversion