Illusion is not Forecasting
Every morning before he left home the illusionist James Randi used to take a piece of paper and write on it “I James Randi will die today”. He’d then sign and date it and slip it in the pocket of his jacket. Had he died, of course, the world would still be full of credulous believers insisting that he was a genuine psychic seer. We smart investors laugh at such fools, of course.
We shouldn’t, though, because every day we’re the victims of people just as clever as Randi and without any of his good moral sense. Every day, across the world, people forecast the unforecastable and predict that markets will boom, or bust, or stagger sideways like a drunken sailor. Eventually one of their predictions comes true and gullible people everywhere equate this with foresight when, in fact, the forecaster has simply been slipping a note in their pocket each morning. In a world where everyone predicts everything occasionally someone’s going to be right.
The Cowles Study
The evidence that most forecasters are simply practising the illusionist’s sleight of hand has been mounting for a very long time. Way back in 1933 Alfred Cowles published a paper on Stock Market Forecasting which “disclosed no evidence of skill in forecasting”. When he extended this study in the 1940’s he commented:
Cowles’ research used the forecasts as a basis for actually making investment decisions – so a completely bullish forecast was assumed to cause an investor to invest 100% of their funds in the market. Computing the compounded results of acting on these forecasts led to a series of observations, the gist of which is that the forecasters exhibited no aptitude for forecasting. Indeed, they underperformed the market and were consistently, by a factor of 4 to 1, on the bullish side:
Given that markets were down around as much as they were up in the period under study and that forecasters were nearly always bullish it’s fairly easy to see that they’d be right about 50% of the time. Which is what Cowles found. In fact if you take a view that most forecasters are always bullish you can take a view that most forecasters will look like geniuses in a bull market and bozos in a bear market without exhibiting any particular skill in either direction. Similarly the smaller number of forecasters who are usually bearish will obtain the reverse position. To whit: forecasters forecast nothing, they’re simply signing their death certificate each morning.
In the current environment, therefore, we might expect to see a few consistently bearish commentators looking as though they can outwit the markets. Nouriel Roubini, aka Dr. Doom, has recently pointed out that his s track record of forecasting major moves in the recent past is very good – having called five out of six movements correctly. Only the five called correctly were all downwards and the other one was the mother of all rebounds.
Roubini is, by far, one of the most intelligent commentators on markets, but the point is that trying to figure out whether any recent record of success is attributable to skill or luck is pretty much impossible. So the question is: if forecasters are so useless why do people keep on using them?
The Seer-Sucker Theory
J. Scott Armstrong has come up with The Seersucker Theory to explain this. What he showed, through a wide range of examples taken from finance, psychology, medicine, etc was that forecasting success is not generally related to expertise over and above a pretty minimal level of competance In fact forecasting accuracy seems to drop once people get above a certain level of expertise. It’s not entirely obvious why this is but there are hints that confirmation bias is involved: seers are less likely to look for disconfirming evidence to cross-check their opinions than less confident semi-experts.
Meanwhile the seer-sucker theory states:
Herds of Analysts
However, Hong, Kubik and Solomon in Security Analyst’s Career Concerns and Herding of Investment Forecasts have proposed an interesting variation of why analyst forecasts aren’t very good in general. Herding – the behavioural trait that causes investors to move in a given direction at the same time – is triggered by career related incentives. This, of course, is not generally a factor for private investors and therefore is often neglected in considerations about why professionals make their recommendations.
What they find is that younger analysts tend to herd more than their more experienced colleagues: less experienced analysts tend to be punished more heavily for getting their forecasts wrong so they have every incentive to stick with the crowd. In contrast older analysts, who have presumably built up their reputations, face less risk of termination. Basically if a younger analyst makes a bold forecast and gets it wrong they’re likely to lose their job, while doing so and getting it right seems to make little difference to their immediate career prospects.
Although the study implies that the more experienced analysts are more accurate forecasters it, unfortunately, doesn’t make clear whether they do any better than chance over long periods. If Cowles’ findings still stack up the probability is not. Either way relying on consensus forecasts by great stampeding herds of perversely incentivised analysts isn’t likely to yield great results. Look for the outliers and then analyse them: still no free lunches, just hints and intimations.
Give Me Stories, Not Facts
Of course, in writing this, I’m well aware that forecasters will carry on forecasting and followers will carry on following: McKinsey have just published another study confirming nothing's changed. When the inventors of the crop-circle came forward to explain how they’d fooled scientists the world over with a bit of ply-wood they were almost universally ignored. James Randi has made a second career out of debunking mystical charlatans who prey on people’s gullibility. Yet still they believe and often react violently to having their illusions shattered. It’s as though we can’t grow up and face the real world: it seems that fairies, dragons, magic and mysticism are so much more attractive than the grim reality of credit card repayments.
And this is the hidden truth – we find stories more congenial than facts. Markets are driven less by fundamentals and more by tales of derring-do. Forecasters are story-tellers not diviners of the future. Like all authors of fiction we should enjoy them for what they are: just don’t confuse their narratives with proper investing.
Related articles: Investing With a Time Machine, Investment Forecasts: Known Unknowns, Real Fortune Telling
Every morning before he left home the illusionist James Randi used to take a piece of paper and write on it “I James Randi will die today”. He’d then sign and date it and slip it in the pocket of his jacket. Had he died, of course, the world would still be full of credulous believers insisting that he was a genuine psychic seer. We smart investors laugh at such fools, of course.
We shouldn’t, though, because every day we’re the victims of people just as clever as Randi and without any of his good moral sense. Every day, across the world, people forecast the unforecastable and predict that markets will boom, or bust, or stagger sideways like a drunken sailor. Eventually one of their predictions comes true and gullible people everywhere equate this with foresight when, in fact, the forecaster has simply been slipping a note in their pocket each morning. In a world where everyone predicts everything occasionally someone’s going to be right.
The Cowles Study
The evidence that most forecasters are simply practising the illusionist’s sleight of hand has been mounting for a very long time. Way back in 1933 Alfred Cowles published a paper on Stock Market Forecasting which “disclosed no evidence of skill in forecasting”. When he extended this study in the 1940’s he commented:
“The wording of many of the forecasts is indefinite, and it would frequently be possible for the forecaster after the event to present a plausible argument in favor of an interpretation other than the one made by the reader”.Gosh, do tell.
Cowles’ research used the forecasts as a basis for actually making investment decisions – so a completely bullish forecast was assumed to cause an investor to invest 100% of their funds in the market. Computing the compounded results of acting on these forecasts led to a series of observations, the gist of which is that the forecasters exhibited no aptitude for forecasting. Indeed, they underperformed the market and were consistently, by a factor of 4 to 1, on the bullish side:
“The persistent and unwarranted record of optimism can possibly be explained on the grounds that readers prefer good news to bad, and that a forecaster who presents a cheerful point of view thereby attracts a following without which he would probably be unable to remain long in the business of forecasting”.A 50-50 Bet
Given that markets were down around as much as they were up in the period under study and that forecasters were nearly always bullish it’s fairly easy to see that they’d be right about 50% of the time. Which is what Cowles found. In fact if you take a view that most forecasters are always bullish you can take a view that most forecasters will look like geniuses in a bull market and bozos in a bear market without exhibiting any particular skill in either direction. Similarly the smaller number of forecasters who are usually bearish will obtain the reverse position. To whit: forecasters forecast nothing, they’re simply signing their death certificate each morning.
In the current environment, therefore, we might expect to see a few consistently bearish commentators looking as though they can outwit the markets. Nouriel Roubini, aka Dr. Doom, has recently pointed out that his s track record of forecasting major moves in the recent past is very good – having called five out of six movements correctly. Only the five called correctly were all downwards and the other one was the mother of all rebounds.
Roubini is, by far, one of the most intelligent commentators on markets, but the point is that trying to figure out whether any recent record of success is attributable to skill or luck is pretty much impossible. So the question is: if forecasters are so useless why do people keep on using them?
The Seer-Sucker Theory
J. Scott Armstrong has come up with The Seersucker Theory to explain this. What he showed, through a wide range of examples taken from finance, psychology, medicine, etc was that forecasting success is not generally related to expertise over and above a pretty minimal level of competance In fact forecasting accuracy seems to drop once people get above a certain level of expertise. It’s not entirely obvious why this is but there are hints that confirmation bias is involved: seers are less likely to look for disconfirming evidence to cross-check their opinions than less confident semi-experts.
Meanwhile the seer-sucker theory states:
“No matter how much evidence exists that seers do not exist, suckers will pay for the existence of suckers”.Or: for every seer there’s a sucker. One reason for this might be to do with avoidance of responsibility on behalf of the suckers. All too often in situations involving risk and uncertainty “experts” are called in who have no realistic chance of making a better decision than anyone else: that’s the problem with uncertainty. It’s uncertain. It’s unpredictable.
Herds of Analysts
However, Hong, Kubik and Solomon in Security Analyst’s Career Concerns and Herding of Investment Forecasts have proposed an interesting variation of why analyst forecasts aren’t very good in general. Herding – the behavioural trait that causes investors to move in a given direction at the same time – is triggered by career related incentives. This, of course, is not generally a factor for private investors and therefore is often neglected in considerations about why professionals make their recommendations.
What they find is that younger analysts tend to herd more than their more experienced colleagues: less experienced analysts tend to be punished more heavily for getting their forecasts wrong so they have every incentive to stick with the crowd. In contrast older analysts, who have presumably built up their reputations, face less risk of termination. Basically if a younger analyst makes a bold forecast and gets it wrong they’re likely to lose their job, while doing so and getting it right seems to make little difference to their immediate career prospects.
Although the study implies that the more experienced analysts are more accurate forecasters it, unfortunately, doesn’t make clear whether they do any better than chance over long periods. If Cowles’ findings still stack up the probability is not. Either way relying on consensus forecasts by great stampeding herds of perversely incentivised analysts isn’t likely to yield great results. Look for the outliers and then analyse them: still no free lunches, just hints and intimations.
Give Me Stories, Not Facts
Of course, in writing this, I’m well aware that forecasters will carry on forecasting and followers will carry on following: McKinsey have just published another study confirming nothing's changed. When the inventors of the crop-circle came forward to explain how they’d fooled scientists the world over with a bit of ply-wood they were almost universally ignored. James Randi has made a second career out of debunking mystical charlatans who prey on people’s gullibility. Yet still they believe and often react violently to having their illusions shattered. It’s as though we can’t grow up and face the real world: it seems that fairies, dragons, magic and mysticism are so much more attractive than the grim reality of credit card repayments.
And this is the hidden truth – we find stories more congenial than facts. Markets are driven less by fundamentals and more by tales of derring-do. Forecasters are story-tellers not diviners of the future. Like all authors of fiction we should enjoy them for what they are: just don’t confuse their narratives with proper investing.
Related articles: Investing With a Time Machine, Investment Forecasts: Known Unknowns, Real Fortune Telling
Very nice as always.
ReplyDeleteTypo: "a pretty minimla level. of competance In fact forecasting"
And this is the hidden truth – we find stories more congenial than facts.
ReplyDeleteIt's not because we are dummies. Stories are often more true than facts. Facts can be easily faked. Stories that hit us on a deep level often do so because they express important truths.
The idea that the market cannot be forecasted is every bit as much of a story as the idea that it cannot be. Is there anyone who truly believes that the market cannot be forecasted? If that were so, returns would be random and stock investing would be pure gambling. And we all gambling with our retirement money?
We all know on some level of consciousness that some sorts of market forecasts work and some do not. We should be directing our energies to distinguish the one type of forecast from the other. Instead, we try to persuade ourselves that no forecasts work. The message never takes because it defies common sense. Stock returns cannot be random and any non-random phenomena can be forecasted to some extent.
Rob
True.
ReplyDeleteBut this is how humans (and most other animals probably) work. We are not machines working with data and 'facts' and we do not operate in the 'real world'. Everything we see and do and touch and feel is a model built by the brain and nervous system, tied together and given meaning over time by the narratives we build through experience.
Storytelling is an inevitable part of that. Sure, some of the more obviously false stories will hopefully fade, like fairies and ghosts, but saying "I think there will probably be a double dip" is not obviously false and so will likely always be with us (spot the hidden forecast).