Hitting a Barn Door
Take a rifle and randomly spray bullets at the side of a barn. Invite some gun-toting friends around to see your handiwork and accept their lavish praise as a dead-eyed sharpshooter, knowing all the while it’s an illusion. The trick is to paint the targets after you’ve made the bullet holes.
This, the Sharpshooter Effect, is essentially how many business gurus and investment analysts make their living. Worse, the effect affects statistical analysis of data that are genuinely important, like clusters of birth defects. The problem is that we’re not very good at distinguishing randomness from order and this causes us no end of grief as we pursue illusory patterns in the belief that we’re being smart.
Meeting Oprah
Management and investment gurus are a breed apart whose opinions offer the rest of us hope that we can learn from their ideas to develop skills that ensure we can create above average companies or secure above average returns on our investments. In book after book and article after article we’re treated to example after example of real-life imitating theory as the gurus’ ideas are hammered home to us in the way we understand best; one damn anecdote at a time.
Of course, management gurus are out to make a living and writing books entitled “I Don’t Know Any More Than You Do” and “Duh?” isn’t a surefire way to make the best-seller lists and meet Oprah. In the main, though, all these erstwhile peddlers of secondhand hope are doing is free-riding on the Sharpshooter Effect, by painting targets around their examples to “prove” whatever point they’re trying to make. Of course, they never point out the other examples where they fall over pulling the trigger, shoot the weathercock and end up flat on their backs in the cattle trough with a chicken on their heads.
Non-Random Humans
The trouble is that the world is just so full of stuff that if you look hard enough you’ll be bound to find some correlations. Unfortunately, unless you do proper statistical analysis, the likelihood is that they’ll be illusory. For example, clusters of early childhood cancer sufferers in specific geographic areas, recently focused on the presence of mobile phone antennae, are regularly shown to be probably nothing more than a normal statistical variation caused by arbitrarily drawing targets around the clusters after the event. Not that the studies showing this convince anyone, of course.
The problem is that randomness is, well, both much more and much less random than we think. A very large but truly random set of 1’s and 0’s will contain several sequences of hundreds of successive 1’s (and 0’s). Try persuading anyone that that’s random. This problem leads to a class of illusions called clustering biases, of which the Sharpshooter Effect is one, where we perceive order in what are simply statistically probable clusters of similar results.
So if our favourite stock goes up for twenty successive days we’re more likely to attribute this to the fact that there’s something going on than to the inevitability of some stock going up for twenty successive days no matter what in the world’s occurring. We pull these patterns out of the data, paint targets around them and then wonder why we don’t make any money in the long term.
Control Freaks
Underlying this, though, are some psychological mechanisms that really aren’t fully understood. Tversky and Kahneman argued that we’re fooled by the representative heuristic, an illusion where we compare the topic under analysis with whatever data we can easily bring to mind – a side-effect of the availability heuristic. The problem is, of course, that a lifetime simply isn’t long enough to generate enough data to do a proper comparison and when we engage in this kind of pattern matching we fall prey to the law of small numbers, an idea also coined by Tversky and Kahneman as a counterpoint to the law of large numbers.
The latter, which is a genuine statistical effect, states that as the number of samples of a specific behaviour increases then you’re more and more likely to home in on the average sample rates. So, for example, if you toss a coin a thousand times you’re more likely to get closer to the 50/50 ratio we’d expect than if you toss it a hundred times.
Sarcastic Streaks
The law of small numbers, however, is a sarcastic reference to its genuine progenitor – the idea that people generate illusory correlations based on very small samples. In the coin tossing example, you could easily end up with ten heads in a sample of ten coin tosses. Extrapolating from this wouldn’t give you much of an idea of the true probabilities involved, but is exactly what we’re evolved to do – after all, we don’t usually get hundreds of chances to observe a specific behaviour. As the authors remark:
However, we can also end up basing our opinions on a sample of one. In essence we remember and are influenced by exceptions to the norm, not by the norm itself. As Risen, Gilovich and Dunning showed in One Shot Illusory Correlations and Stereotype Formation:
Predictions in Hindsight
Taken together our ability to generate illusory correlations based on limited data leads us into all sorts of inept behaviour when it comes to investing. Invariably you can find someone willing to explain, after the event, why some stock has moved 0.25% on a given day, but the amount of credibility we should give such explanations is approximately zero, to as many significant figures as you wish to add. Our credulity in the face of such randomness is, however, an ongoing and apparently ineradicable feature of the human condition.
Trying to find explanations for random behaviour is, however, more or less the entire rationale behind the profession of business and investment gurus who make a living expounding on various theories of dubious quality. In the main these rely on a single, simple human characteristics – the unfailing accuracy of the Texas sharpshooter.
The eponymous gunman was renowned for his accuracy in hitting targets drawn on the side of a barn. Sadly, like all great magicians, the skill was in the illusion, not in the shooting. We need to learn to mistrust anyone drawing conclusions from selective data (which is, of course, a dangerous bit of stone throwing from a glasshouse dweller) and to stop worrying about lack of control of our investments. Better go do something useful, like free-form mountain climbing or chainsaw juggling, where we have some genuine element of control.
Related Articles: Monty Hall Economics, Recency, Hot Hands and the Gambler's Fallacy, The Lottery of Stockpicking
Take a rifle and randomly spray bullets at the side of a barn. Invite some gun-toting friends around to see your handiwork and accept their lavish praise as a dead-eyed sharpshooter, knowing all the while it’s an illusion. The trick is to paint the targets after you’ve made the bullet holes.
This, the Sharpshooter Effect, is essentially how many business gurus and investment analysts make their living. Worse, the effect affects statistical analysis of data that are genuinely important, like clusters of birth defects. The problem is that we’re not very good at distinguishing randomness from order and this causes us no end of grief as we pursue illusory patterns in the belief that we’re being smart.
Meeting Oprah
Management and investment gurus are a breed apart whose opinions offer the rest of us hope that we can learn from their ideas to develop skills that ensure we can create above average companies or secure above average returns on our investments. In book after book and article after article we’re treated to example after example of real-life imitating theory as the gurus’ ideas are hammered home to us in the way we understand best; one damn anecdote at a time.
Of course, management gurus are out to make a living and writing books entitled “I Don’t Know Any More Than You Do” and “Duh?” isn’t a surefire way to make the best-seller lists and meet Oprah. In the main, though, all these erstwhile peddlers of secondhand hope are doing is free-riding on the Sharpshooter Effect, by painting targets around their examples to “prove” whatever point they’re trying to make. Of course, they never point out the other examples where they fall over pulling the trigger, shoot the weathercock and end up flat on their backs in the cattle trough with a chicken on their heads.
Non-Random Humans
The trouble is that the world is just so full of stuff that if you look hard enough you’ll be bound to find some correlations. Unfortunately, unless you do proper statistical analysis, the likelihood is that they’ll be illusory. For example, clusters of early childhood cancer sufferers in specific geographic areas, recently focused on the presence of mobile phone antennae, are regularly shown to be probably nothing more than a normal statistical variation caused by arbitrarily drawing targets around the clusters after the event. Not that the studies showing this convince anyone, of course.
The problem is that randomness is, well, both much more and much less random than we think. A very large but truly random set of 1’s and 0’s will contain several sequences of hundreds of successive 1’s (and 0’s). Try persuading anyone that that’s random. This problem leads to a class of illusions called clustering biases, of which the Sharpshooter Effect is one, where we perceive order in what are simply statistically probable clusters of similar results.
So if our favourite stock goes up for twenty successive days we’re more likely to attribute this to the fact that there’s something going on than to the inevitability of some stock going up for twenty successive days no matter what in the world’s occurring. We pull these patterns out of the data, paint targets around them and then wonder why we don’t make any money in the long term.
Control Freaks
Underlying this, though, are some psychological mechanisms that really aren’t fully understood. Tversky and Kahneman argued that we’re fooled by the representative heuristic, an illusion where we compare the topic under analysis with whatever data we can easily bring to mind – a side-effect of the availability heuristic. The problem is, of course, that a lifetime simply isn’t long enough to generate enough data to do a proper comparison and when we engage in this kind of pattern matching we fall prey to the law of small numbers, an idea also coined by Tversky and Kahneman as a counterpoint to the law of large numbers.
The latter, which is a genuine statistical effect, states that as the number of samples of a specific behaviour increases then you’re more and more likely to home in on the average sample rates. So, for example, if you toss a coin a thousand times you’re more likely to get closer to the 50/50 ratio we’d expect than if you toss it a hundred times.
Sarcastic Streaks
The law of small numbers, however, is a sarcastic reference to its genuine progenitor – the idea that people generate illusory correlations based on very small samples. In the coin tossing example, you could easily end up with ten heads in a sample of ten coin tosses. Extrapolating from this wouldn’t give you much of an idea of the true probabilities involved, but is exactly what we’re evolved to do – after all, we don’t usually get hundreds of chances to observe a specific behaviour. As the authors remark:
"The true believer in the law of small numbers commits his multitude of sins against the logic of statistical inference in good faith. The representation hypothesis describes a cognitive or perceptual bias, which operates regardless of motivational factors".Basically we're statistically stupid, but for good evolutionary reasons.
However, we can also end up basing our opinions on a sample of one. In essence we remember and are influenced by exceptions to the norm, not by the norm itself. As Risen, Gilovich and Dunning showed in One Shot Illusory Correlations and Stereotype Formation:
"A single instance of unusual behavior by a member of a rare group is sufficient to create and association between group and behavior".Essentially we can create stereotypes about people we know nothing about based on virtually no data whatsoever. Then we go throw rocks at them.
Predictions in Hindsight
Taken together our ability to generate illusory correlations based on limited data leads us into all sorts of inept behaviour when it comes to investing. Invariably you can find someone willing to explain, after the event, why some stock has moved 0.25% on a given day, but the amount of credibility we should give such explanations is approximately zero, to as many significant figures as you wish to add. Our credulity in the face of such randomness is, however, an ongoing and apparently ineradicable feature of the human condition.
Trying to find explanations for random behaviour is, however, more or less the entire rationale behind the profession of business and investment gurus who make a living expounding on various theories of dubious quality. In the main these rely on a single, simple human characteristics – the unfailing accuracy of the Texas sharpshooter.
The eponymous gunman was renowned for his accuracy in hitting targets drawn on the side of a barn. Sadly, like all great magicians, the skill was in the illusion, not in the shooting. We need to learn to mistrust anyone drawing conclusions from selective data (which is, of course, a dangerous bit of stone throwing from a glasshouse dweller) and to stop worrying about lack of control of our investments. Better go do something useful, like free-form mountain climbing or chainsaw juggling, where we have some genuine element of control.
Related Articles: Monty Hall Economics, Recency, Hot Hands and the Gambler's Fallacy, The Lottery of Stockpicking
Our credulity in the face of such randomness is, however, an ongoing and apparently ineradicable feature of the human condition.
ReplyDeleteExcept not at this blog!
And, if not at this blog, not anywhere! (once the rest of the world catches up to this blog).
We didn't know how to make fire. Then we did.
We didn't know how to make the wheel. Then we did.
We thought Perry Como was it. Then the Beatles came along.
Thing change. People learn. It's a process and we are into that process today. The very fact that you talk about the realities proves it.
The only thing holding us back today is our lack of courage in acknowledging how stupid we have been and for how long. But we'll get over that.
We don't have any choice. We're running out of money! Everybody is yelling and stuff. We'll take action eventually if only to stop the yelling.
Rob