Can You Forecast Better than a Dart Throwing Chimp?

The fox knows many little things, but the hedgehog knows one big thing - Archilochus

Cover Story

Philip Tetlock has spent many years studying the ability of experts to predict important events in their sphere of interest and come to the not entirely astonishing conclusion that they aren't much good at it. Soviet experts, for instance, missed the possibility that the Soviet Union might stop existing. On the other hand, they're exceptionally good at promoting themselves: being wrong is no impediment to fame, it seems.

Tetlock has discovered that people fall into two groups that he labels 'hedgehogs' and 'foxes', Hedgehogs have one big idea and tend to interpret the world in terms of it. Foxes have lots of ideas and are more flexible in the face of change. Unsurprisingly foxes are better at predicting stuff; but does that make them better investors?

Talking Heads

Tetlock's research uncovered some interesting things about the way expert opinion is presented through the media. For a start the experts who are most popular with the press are the worst at making predictions. If we were to assume the same is true with financial expertise it suggests that the most popular talking heads are the ones that we should ignore at all costs.

When he dug into the data and started analysing personality traits Tetlock discovered a difference in cognitive styles between the popular yet woefully incompetent group and the less popular but more accurate experts. The former he labelled hedgehogs and the latter foxes, taking the terms from Isaiah Berlin's essay on Tolstoy, The Hedgehog and the Fox. 

Broadly speaking, foxes have lots of little ideas, are happy to deal with uncertainty and nuances and regard dissenting opinions as an opportunity to learn. Hedgehogs, on the other hand, have fixity of purpose and ideas and believe that these ideas are almost akin to physical laws. They seem to thrive in areas that don't really admit of certainty: cf. Skinner in psychology, Marx in economic theory, Laplace on the Bell curve and nearly every writer with a Big Idea about something.

Fox v Hedgehog

Beginning with the idea that foxes were better at making predictions than hedgehogs Tetlock started looking at the underlying differences in cognitive approach and found a number of quite clear differences:

1. Foxes are cautious about making predictions. Hedgehogs are not cautious about making predictions. Hedgehogs suffer from overconfidence and hindsight bias.
2. Foxes are avid gatherers of ideas from everywhere. Hedgehogs tend to specialize and resent outside opinions or ideas: they suffer from what Charlie Munger calls "man with a hammer syndrome".
3. Foxes follow Keynes - if something isn't working they look for a new idea or approach. Hedgehogs  follow Friedman and stick to their approach, using new data to tweak their existing models.
4. Foxes are willing to accept they're wrong. Hedgehogs blame errors on anything other than themselves. In fact, quite often, they refuse to accept they were wrong even when they clearly are.
5. Foxes think the world is a complex place. Hedgehogs think that if you can strip away all the noise there's underlying simplicity (think Good versus Evil).
6. Foxes are more concerned with practical data than theory, Hedgehogs tend to regard empirical data as noise obscuring the real, underlying truth. 
7. Foxes are attractive and interesting people. Hedgehogs are annoying ideologues who pin you in a corner at parties and bore you senseless.

(OK, I made that last one up).

Experience And Bust

Despite all of this it still seems surprising that Tetlock has found that, with experience, foxes get better at forecasting but hedgehogs get worse. The evidence is buried in the small print in his book Expert Political Judgment, but ends with the damning (albeit enjoyable) quote:

"[Expert] hedgehogs and the dart-throwing chimp had equivalent forecasting skill"

This is, perhaps, a partial explanation for some of the other evidence we've seen suggesting that experience doesn't always improve investor performance. Generally the research doesn't distinguish between hedgehogs and foxes, and the blending of the two approaches will presumably obscure underlying differences. There's a research project in there somewhere for an enterprising young academic.

Hedgehog Evolution

Despite all of this Tetlock points out that being a hedgehog is often advantageous:

Foxes were better equipped to survive in rapidly changing environments in which those who abandoned bad ideas quickly held the advantage. Hedgehogs were better equipped to survive in static environments that rewarded persisting with tried-and-true formulas. Our species—homo sapiens—is better off for having both temperaments.

And, of course, financial markets are an analogue of the real world. They go through long periods of similar behavior, where a consistent investing style will yield big returns - periods in which hedgehog type approaches will either win big (if correctly aligned) or lose equally big (if not). Foxes will underperform successful hedgehogs, who will be very visible (yet scarce), and outperform unsuccessful ones, who will be invisible (and prevalent).

Come the turn of the market foxes will win - and previously successful hedgehogs will lose big, because they'll continue to believe that they're right and the markets are wrong. Eventually a new consensus will emerge, and a new generation of successful hedgehogs will arise - and so the cycle continues.

Self-doubt, banished

Tetlock is fascinating on the way in which these different cognitive styles impact on the major behavioral biases. Foxes are less impacted by overconfidence and hindsight bias, but perhaps more inclined to prevaricate on decision making. Sometimes there's such a thing as too much benefit of the doubt.

Expert hedgehogs are rarely assailed by self-doubt and often refuse to consider new data that they consider ridiculous because it sits outside of their internalized models. And sometimes they're right and sometimes they're wrong. But, as Tetlock points out, always they're certain that they're right, even in the face of obvious error:

Just how overwhelming evidence must be to break this barrier is illustrated by the ridiculously high thresholds of proof that partisans set for conceding their side did something scandalous. It required the Watergate recordings to force Nixon defenders to acknowledge that he had obstructed justice, and it required DNA testing of Monica Lewinski’s dress to compel Clinton defenders to concede that something improper had occurred 

Equally hedgehogs will seize on the vaguest support for their ideas and trumpet it for all it's worth. This asymmetry in the standards of evidence needed is usually all the evidence you should need to stay well clear of a fast talking hedgehog.

Probabilistic Thinking

The critical thing about investing - or any other complex system that doesn't admit of certainty - is to recognize all of the possible outcomes and assign relative probabilities to them. You may be absolutely sure that the Eurozone will survive the next crisis or that the Fed will rescue the banks the next time they screw up - but you'd do well to consider the possibility that those certainties are misplaced.

And that's why foxes will, on average and over time, get better results than hedgehogs. They'll avoid the mistakes that come from unwarranted certainty - overconfidence, as we better know it. They think small, and they think wide. Of course, at any given time there'll be a few narrowly focused hedgehogs producing stunning performance - but they're the statistical outliers. If all you ever do is take extreme positions you're more likely to fail than succeed.

The Exceptions

Despite all of the above any fox would be wise to consider the possibility that a hedgehog (or a chimp) may sometimes be correct. In the lead up to the Second World War most British politicians, many scarred by memories of the Great War, discounted the voices dissenting from negotiations with Hitler. With good reason, for the dissenters were led by a man with all the spiky tendencies of a hedgehog, and a history of terrible decisions.

Yet history reveres him, and the last survey of its kind listed him the Greatest Ever Briton. Cometh the moment, cometh the man - and the man (or woman) is nearly always a hedgehog. Sometimes you have to choose the right hedgehog to follow. As the man himself said:

However beautiful the strategy, you should occasionally look at the results.

(Winston Churchill)

  

Related articles:

• Clueless: Meet The Overprecise Experts
• Whither Forecasting? The Butterfly Stirs ...
• Investor Decisions –Experience Is Still Not Enough (But It Helps A Bit)
• H is for Hndsight Bias 
• Overconfidence and Overoptimism
• Newton's Financial Crisis: The Limits of Quantification
• Where Two Strangers Never Meet: Self-Serving Bias
• A Keynesian Theory of Mind