Take the Cognitive Reflection Test
Answer the following questions and then read on.
(1) A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? _____ cents
(2) If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? _____ minutes
(3) In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? _____ days
Intuitively Obvious – and Wrong
Each of the questions on the Cognitive Reflection Test has an immediate, intuitive answer – which happens to be completely wrong. If you answered, or even considered, any of 10 cents, 100 minutes or 24 days you’re in good company (yours truly included). Out of 3428 people tested in the original research a third got all three answers wrong while only 17% got all correct. Even many of those who got the answers right had to think at least twice.
These particular questions are the mental equivalent of optical illusions. They appear to be easy, but aren’t. Tests of similar complexity which are more obviously difficult yield more correct answers so it’s not simply a question of basic maths skills. Something else is going on here. Something, it turns out, that may completely screw up psychology’s ideas about the way people think about finance.
Dumb or Smart?
When the respondants were split into groups dependent on how well they’d done on the CRT and (painlessly) probed for underlying differences what was found, broadly, is that those who did well are generally smart and those who did badly are generally dumb. Which is the psychologist’s equivalent of discovering that gravity makes things fall by dropping dumbbells on your feet.
However, when the “smart” and “dumb” groups were investigated further something genuinely interesting showed up. The “dumb” group turned out to behave exactly like behavioural finance predicts but the “smart” group didn’t. Which is a bit of a problem for the economists who’ve spent thirty years developing models for making money which rely on everyone acting dumb. It’s roughly the equivalent of discovering that gravity doesn’t work the same way everywhere; which can be quite disconcerting if your dumbells suddenly float off into the distance.
The Findings of the CRT
There were two main findings from the CRT. The first is that the “smarter” people were better at discounting time. Or, in lay language, they’re prepared to wait longer for a larger reward rather than taking a small, certain amount immediately – as long as the odds favour them. However, this only applies in the short-term – once timelines stretch out this effect disappears, which is also sensible, since the longer you have to wait for your delayed reward the more likely it is to never occur.
It’s the second finding that’s really interesting, though. Prospect Theory, the cornerstone of behavioural finance describing people’s risk taking behaviour, predicts that people are risk adverse when protecting a gain and risk takers when chasing a loss. This was spectacularly true of the low CRT scoring group, but not true at all of the high scoring group – suggesting that cognitive ability, aka I.Q., is critical in evaluation of decision making theories.
Or, to put it bluntly, behavioural finance is wrong.
What Do I.Q. Tests Measure?
Shane Frederick, the researcher, compared respondents’ CRT scores with results from other I.Q. tests and discovered a good but not perfect correlation. The CRT is measuring something similar, but not precisely the same as these other tests. But what do I.Q. tests measure anyway?
The founder of I.Q. testing, Alfred Binet, developed the concept to track improvements in learning, not absolute levels of intelligence. Scientists to a man (and they’re nearly always male) have proceeded to ignore him and spent the century or so since drawing bell curves to “prove” their preconceived hypotheses.
It was the U.S. Army in World War 1 that picked up the idea as a general measure for recruits and it’s been downhill ever since. Stephen Jay Gould in The Mismeasure of Man lays out the whole sorry story. It’s almost impossible to quickly convey exactly how manipulated the idea of I.Q. tests has been, but the concept keeps on rearing its ugly head like a particularly demented, stake-proof, garlic-loving vampire. By way of example, in the U.K. the academic Cyril Burt’s separated twin studies proved that genes are more important than upbringing in intelligence – i.e. that intelligence is innate, not learned. His findings influenced half a century of British education so it was just a tiny little bit of a shame that it turned out he’d faked his results.
So we can say, fairly safely, that whatever the CRT is measuring it’s not exactly intelligence. Frederick argues that it’s showing something he calls “cognitive reflection”—the ability or disposition to resist reporting the response that first comes to mind. If correct, why should this matter so much?
Does the CRT Invalidate Prospect Theory?
Prospect Theory tells us something about how people make decisions under conditions of uncertainty. The new research is telling us that some, thoughtful, people will make different decisions from those predicted by the theory and that we can identify these people, ahead of time, by using the CRT.
Follow up work from Frederick has indeed shown significant differences in risk taking behaviour amongst low and high CRT groups. Which would you prefer – the certainty of $500 now or a 15% chance of $1 million?
High CRT scorers overwhelmingly went for the gamble – 82% of them chose it while only a quarter of low CRT scorers gambled. This leads us straight into the quagmire of moral relativism: who decides what a “better” choice is and who is to say that the higher CRT scorers are making “better” decisions than the lower CRT scorers?
Dumb Money
Well, I will for one. Choosing five hundred bucks over a better than 1 in 7 chance of a life changing amount of money would be a totally bloody stupid decision no matter how poor, ignorant and desperate you are.
Still, this is all theoretical and we just don’t know how this would work in real situations. We’ve seen a similar proposal before from John A. List, whose staged but naturalistic experiments on trading behaviour suggested that experienced traders could overcome the loss aversion traits shown in the original research. Since then, as we’ve also seen, a completely naturalistic study using professional golfers has provided support for the original findings of Prospect Theory, throwing doubt on whether List’s experiment is really replicating real-world conditions or is somehow generating something even more artificial.
Separating genetic capabilities and learned ones is more like unbaking a cake than unwinding tangled threads. However, if the CRT tells us anything about investing it’s that we need to think hard and think properly and to take our time about it. Such skills can be learned and there are techniques investors can use to help develop them: just as long as they recognise they need them in the first place, of course.
Oh Yeah, The Results
The Cognitive Reflection Test (CRT) was developed by Professor Shane Frederick as described in Cognitive Reflections and Decision Making.
The correct answers are:
Related Articles: Loss Aversion Affects Tiger Woods, Too, The Death of Homo economicus, Pascal’s Wager – For Richer, For Poorer
Answer the following questions and then read on.
(1) A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? _____ cents
(2) If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? _____ minutes
(3) In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? _____ days
Intuitively Obvious – and Wrong
Each of the questions on the Cognitive Reflection Test has an immediate, intuitive answer – which happens to be completely wrong. If you answered, or even considered, any of 10 cents, 100 minutes or 24 days you’re in good company (yours truly included). Out of 3428 people tested in the original research a third got all three answers wrong while only 17% got all correct. Even many of those who got the answers right had to think at least twice.
These particular questions are the mental equivalent of optical illusions. They appear to be easy, but aren’t. Tests of similar complexity which are more obviously difficult yield more correct answers so it’s not simply a question of basic maths skills. Something else is going on here. Something, it turns out, that may completely screw up psychology’s ideas about the way people think about finance.
Dumb or Smart?
When the respondants were split into groups dependent on how well they’d done on the CRT and (painlessly) probed for underlying differences what was found, broadly, is that those who did well are generally smart and those who did badly are generally dumb. Which is the psychologist’s equivalent of discovering that gravity makes things fall by dropping dumbbells on your feet.
However, when the “smart” and “dumb” groups were investigated further something genuinely interesting showed up. The “dumb” group turned out to behave exactly like behavioural finance predicts but the “smart” group didn’t. Which is a bit of a problem for the economists who’ve spent thirty years developing models for making money which rely on everyone acting dumb. It’s roughly the equivalent of discovering that gravity doesn’t work the same way everywhere; which can be quite disconcerting if your dumbells suddenly float off into the distance.
The Findings of the CRT
There were two main findings from the CRT. The first is that the “smarter” people were better at discounting time. Or, in lay language, they’re prepared to wait longer for a larger reward rather than taking a small, certain amount immediately – as long as the odds favour them. However, this only applies in the short-term – once timelines stretch out this effect disappears, which is also sensible, since the longer you have to wait for your delayed reward the more likely it is to never occur.
It’s the second finding that’s really interesting, though. Prospect Theory, the cornerstone of behavioural finance describing people’s risk taking behaviour, predicts that people are risk adverse when protecting a gain and risk takers when chasing a loss. This was spectacularly true of the low CRT scoring group, but not true at all of the high scoring group – suggesting that cognitive ability, aka I.Q., is critical in evaluation of decision making theories.
Or, to put it bluntly, behavioural finance is wrong.
What Do I.Q. Tests Measure?
Shane Frederick, the researcher, compared respondents’ CRT scores with results from other I.Q. tests and discovered a good but not perfect correlation. The CRT is measuring something similar, but not precisely the same as these other tests. But what do I.Q. tests measure anyway?
The founder of I.Q. testing, Alfred Binet, developed the concept to track improvements in learning, not absolute levels of intelligence. Scientists to a man (and they’re nearly always male) have proceeded to ignore him and spent the century or so since drawing bell curves to “prove” their preconceived hypotheses.
It was the U.S. Army in World War 1 that picked up the idea as a general measure for recruits and it’s been downhill ever since. Stephen Jay Gould in The Mismeasure of Man lays out the whole sorry story. It’s almost impossible to quickly convey exactly how manipulated the idea of I.Q. tests has been, but the concept keeps on rearing its ugly head like a particularly demented, stake-proof, garlic-loving vampire. By way of example, in the U.K. the academic Cyril Burt’s separated twin studies proved that genes are more important than upbringing in intelligence – i.e. that intelligence is innate, not learned. His findings influenced half a century of British education so it was just a tiny little bit of a shame that it turned out he’d faked his results.
So we can say, fairly safely, that whatever the CRT is measuring it’s not exactly intelligence. Frederick argues that it’s showing something he calls “cognitive reflection”—the ability or disposition to resist reporting the response that first comes to mind. If correct, why should this matter so much?
Does the CRT Invalidate Prospect Theory?
Prospect Theory tells us something about how people make decisions under conditions of uncertainty. The new research is telling us that some, thoughtful, people will make different decisions from those predicted by the theory and that we can identify these people, ahead of time, by using the CRT.
Follow up work from Frederick has indeed shown significant differences in risk taking behaviour amongst low and high CRT groups. Which would you prefer – the certainty of $500 now or a 15% chance of $1 million?
High CRT scorers overwhelmingly went for the gamble – 82% of them chose it while only a quarter of low CRT scorers gambled. This leads us straight into the quagmire of moral relativism: who decides what a “better” choice is and who is to say that the higher CRT scorers are making “better” decisions than the lower CRT scorers?
Dumb Money
Well, I will for one. Choosing five hundred bucks over a better than 1 in 7 chance of a life changing amount of money would be a totally bloody stupid decision no matter how poor, ignorant and desperate you are.
Still, this is all theoretical and we just don’t know how this would work in real situations. We’ve seen a similar proposal before from John A. List, whose staged but naturalistic experiments on trading behaviour suggested that experienced traders could overcome the loss aversion traits shown in the original research. Since then, as we’ve also seen, a completely naturalistic study using professional golfers has provided support for the original findings of Prospect Theory, throwing doubt on whether List’s experiment is really replicating real-world conditions or is somehow generating something even more artificial.
Separating genetic capabilities and learned ones is more like unbaking a cake than unwinding tangled threads. However, if the CRT tells us anything about investing it’s that we need to think hard and think properly and to take our time about it. Such skills can be learned and there are techniques investors can use to help develop them: just as long as they recognise they need them in the first place, of course.
Oh Yeah, The Results
The Cognitive Reflection Test (CRT) was developed by Professor Shane Frederick as described in Cognitive Reflections and Decision Making.
The correct answers are:
- 5 cents (not 10 cents)
- 5 minutes (not 100 minutes)
- 47 days (not 24 days).
Related Articles: Loss Aversion Affects Tiger Woods, Too, The Death of Homo economicus, Pascal’s Wager – For Richer, For Poorer
