Models are not Reality
When we model the world we simplify it, because we have to: to model the real-world accurately we’d need to have all the atoms in the universe and a few more. So, the globe of the world revolving gently on my desk afore me, is a highly simplified representation of the planet most of us live on, leaving out what many of us would consider to be most of the important details – stuff like people, beer, football, lively teenage daughters and the latest episode of House. You fill in your own details.
Quants, the purveyors of quantitative investment analysis, also model the world, although in a much more complex, mathematical way. Yet despite the sophistication and elegance of the mathematics they too make simplifications because they have to, and these matter to us all because they’ve brought the world to the point of financial meltdown thrice within a few years.
Models are Metaphors
What quants bring to investment analysis is a set of metaphors about the way the world actually works, rather than any kind of accurate description of what’s really happening. In fact the world is run by such approximations – models of average life expectancy, mean time to failure of aerospace components, the probability of Michael Jackson’s chimp contesting his will – and, in general, these inaccuracies haven’t mattered much to humanity en-masse. However, we’re now in a world dominated by mass computing power and intricately interconnected such that small perturbations in one area can have dramatic ramifications elsewhere.
From the 1980’s, when Portfolio Insurance turned out to be no such thing, to the 1990’s, when Long Term Capital Management became a glorious oxymoron, through to the 2000’s, when the securitised mortgage market turned into a series of financial time bombs in a greater game of pass the ticking parcel, the quantitative models that mathematically illiterate managements have relied on have given no warning of what was about to occur. Yet these models were created by some of the cleverest minds in the world and supported by some of the most serious back-testing that has ever been carried out.
Yet still they failed.
People Are Uncorrelated, Until They Aren’t
What the models failed to capture was that humans don’t behave in simple, predictable and uncorrelated ways. It’s impossible to overstate the importance of the way these models cope with correlation of peoples’ psychology. To sum it up: they don’t. Let me know if that’s too complex an analysis for the mathematical masters of the universe.
Anyone who’s ever been to a nightclub, a football game or even a very loud party will know that there are situations where we don’t act as individuals, buzzing about doing our own thing. These are occasions when we all suddenly stop being individuals and start doing the same thing – usually involving large quantities of drugs and some very bad singing. Although these sorts of events are specifically designed to trigger this behaviour – which is probably a deep evolutionary adaptation to sponsor group behaviour, useful when it comes to running down tasty antelope and dealing with giant, carnivorous sabre toothed beavers – it can also happen in other situations. Most stockmarket booms and busts are generated by similar group effects.
In general, people behave in an uncorrelated fashion right up until the point they don’t. Then we all suddenly do the same thing together. We stop taking flights in the wake of 9/11, we stop letting our children play in the streets because of a single, heavily reported abduction in another country and we start selling our shares because everyone else is. Fear is an awfully big motivator.
If Something Can’t Continue, It Won’t
Quantitative models don’t handle this sudden polarisation of human behaviour very well. Every so often something surprising happens that causes us all to scurry into the nearest hole and the models promptly fall over, usually accompanied by some whizz-kid earnestly explaining that this was a one in a million year event and that it was just bad luck it happened after three years and can he please have his bonus anyway?
Think about the booms and busts in stockmarkets compared to the relatively stable periods in between. In the stable times it’s possible to roughly model the way that people are going to invest, on average, making certain assumptions about the companies comprising the markets and general economic conditions. While these assumptions hold so do the models and this can go on for a long time, long enough to convince people that stability is the norm.
Only stability is not the norm. When the markets enter one of their periodic manic-depressive phases these general models break down – people start to cluster together in fear or greed and do the same thing. Quantitative models work on the basis that the stable times will last forever, but the reality is that they don’t and when they end they do so in highly unexpected and unpredictable ways. Worse still, if any model becomes too popular it will start to influence the real world, to swing the pendulum one way. The trouble with pendulums is that eventually they swing back.
Credit Rating Madness
We can trace the collapse of the banks over the past couple of years to the excessive risks they were taking in holding Collateralised Debt Obligations on mortgage securities, the idea being that if you took a lot of very safe mortgage debt and bundled it up with a bit of really unsafe mortgage debt you’d end up with a safe investment. Whereas, in fact, you ended up with a lot of really unsafe lending to people who were never going to be able to repay their mortgages.
Yet at the time the models investment analysts were relying on simply didn’t identify these risks. Indeed the models of the credit rating agencies were explicitly changed to take into account the quantitative models showing that such securities weren’t risky.
The sheer nuttiness of the credit rating agencies changing their risk models purely because a quantitative model existed that indicated that the risk of these securities collapsing like dominoes in the event of isolated defaults was remote is still hard to believe. It’s not that the models didn’t indicate exactly that. Nor is it that the mathematics behind the models was particularly stupid. Nor was it that the analysts who dreamt up the models were doing anything obviously wrong.
No, it’s none of those things. It’s the fact that a bunch of smart people can possibly believe that any computer model can accurately reflect the real risks in a world dominated by stubborn, irrational, fearful humankind. You’ve got to ask – did none of the overpaid executives running the world’s financial corporations and regulators actually stop to wonder whether someone might have, just possibly, failed to predict everything that might happen in the real world? Did none of them look at the collapse of LTCM and wonder?
We Need Bridge Builders, Not Quants
Well, oddly enough, some of them did. Back in 2006 when the CEO of Citigroup was still dancing in the last chance sub-prime disco the risk management team at Goldman Sachs got together and solemnly inspected their VaR models which told them that risk levels were still low. Then they inspected their brains – and bailed out. Whether that was luck or judgement is still to be decided.
The great Victorian engineers built bridges that endure to this day because they couldn’t exactly model the risks. They built with a margin of safety not with a bonus on margins in mind. Remember this because the quants are not dead, they’re out there yet. They will rise again.
Related Articles: Risky Bankers Need Swiss Cheese, Not VaR, Hedge Funds Ate My Shorts, Black Swans, Tsunamis and Cardiac Arrests
When we model the world we simplify it, because we have to: to model the real-world accurately we’d need to have all the atoms in the universe and a few more. So, the globe of the world revolving gently on my desk afore me, is a highly simplified representation of the planet most of us live on, leaving out what many of us would consider to be most of the important details – stuff like people, beer, football, lively teenage daughters and the latest episode of House. You fill in your own details.
Quants, the purveyors of quantitative investment analysis, also model the world, although in a much more complex, mathematical way. Yet despite the sophistication and elegance of the mathematics they too make simplifications because they have to, and these matter to us all because they’ve brought the world to the point of financial meltdown thrice within a few years.
Models are Metaphors
What quants bring to investment analysis is a set of metaphors about the way the world actually works, rather than any kind of accurate description of what’s really happening. In fact the world is run by such approximations – models of average life expectancy, mean time to failure of aerospace components, the probability of Michael Jackson’s chimp contesting his will – and, in general, these inaccuracies haven’t mattered much to humanity en-masse. However, we’re now in a world dominated by mass computing power and intricately interconnected such that small perturbations in one area can have dramatic ramifications elsewhere.
From the 1980’s, when Portfolio Insurance turned out to be no such thing, to the 1990’s, when Long Term Capital Management became a glorious oxymoron, through to the 2000’s, when the securitised mortgage market turned into a series of financial time bombs in a greater game of pass the ticking parcel, the quantitative models that mathematically illiterate managements have relied on have given no warning of what was about to occur. Yet these models were created by some of the cleverest minds in the world and supported by some of the most serious back-testing that has ever been carried out.
Yet still they failed.
People Are Uncorrelated, Until They Aren’t
What the models failed to capture was that humans don’t behave in simple, predictable and uncorrelated ways. It’s impossible to overstate the importance of the way these models cope with correlation of peoples’ psychology. To sum it up: they don’t. Let me know if that’s too complex an analysis for the mathematical masters of the universe.
Anyone who’s ever been to a nightclub, a football game or even a very loud party will know that there are situations where we don’t act as individuals, buzzing about doing our own thing. These are occasions when we all suddenly stop being individuals and start doing the same thing – usually involving large quantities of drugs and some very bad singing. Although these sorts of events are specifically designed to trigger this behaviour – which is probably a deep evolutionary adaptation to sponsor group behaviour, useful when it comes to running down tasty antelope and dealing with giant, carnivorous sabre toothed beavers – it can also happen in other situations. Most stockmarket booms and busts are generated by similar group effects.
In general, people behave in an uncorrelated fashion right up until the point they don’t. Then we all suddenly do the same thing together. We stop taking flights in the wake of 9/11, we stop letting our children play in the streets because of a single, heavily reported abduction in another country and we start selling our shares because everyone else is. Fear is an awfully big motivator.
If Something Can’t Continue, It Won’t
Quantitative models don’t handle this sudden polarisation of human behaviour very well. Every so often something surprising happens that causes us all to scurry into the nearest hole and the models promptly fall over, usually accompanied by some whizz-kid earnestly explaining that this was a one in a million year event and that it was just bad luck it happened after three years and can he please have his bonus anyway?
Think about the booms and busts in stockmarkets compared to the relatively stable periods in between. In the stable times it’s possible to roughly model the way that people are going to invest, on average, making certain assumptions about the companies comprising the markets and general economic conditions. While these assumptions hold so do the models and this can go on for a long time, long enough to convince people that stability is the norm.
Only stability is not the norm. When the markets enter one of their periodic manic-depressive phases these general models break down – people start to cluster together in fear or greed and do the same thing. Quantitative models work on the basis that the stable times will last forever, but the reality is that they don’t and when they end they do so in highly unexpected and unpredictable ways. Worse still, if any model becomes too popular it will start to influence the real world, to swing the pendulum one way. The trouble with pendulums is that eventually they swing back.
Credit Rating Madness
We can trace the collapse of the banks over the past couple of years to the excessive risks they were taking in holding Collateralised Debt Obligations on mortgage securities, the idea being that if you took a lot of very safe mortgage debt and bundled it up with a bit of really unsafe mortgage debt you’d end up with a safe investment. Whereas, in fact, you ended up with a lot of really unsafe lending to people who were never going to be able to repay their mortgages.
Yet at the time the models investment analysts were relying on simply didn’t identify these risks. Indeed the models of the credit rating agencies were explicitly changed to take into account the quantitative models showing that such securities weren’t risky.
The sheer nuttiness of the credit rating agencies changing their risk models purely because a quantitative model existed that indicated that the risk of these securities collapsing like dominoes in the event of isolated defaults was remote is still hard to believe. It’s not that the models didn’t indicate exactly that. Nor is it that the mathematics behind the models was particularly stupid. Nor was it that the analysts who dreamt up the models were doing anything obviously wrong.
No, it’s none of those things. It’s the fact that a bunch of smart people can possibly believe that any computer model can accurately reflect the real risks in a world dominated by stubborn, irrational, fearful humankind. You’ve got to ask – did none of the overpaid executives running the world’s financial corporations and regulators actually stop to wonder whether someone might have, just possibly, failed to predict everything that might happen in the real world? Did none of them look at the collapse of LTCM and wonder?
We Need Bridge Builders, Not Quants
Well, oddly enough, some of them did. Back in 2006 when the CEO of Citigroup was still dancing in the last chance sub-prime disco the risk management team at Goldman Sachs got together and solemnly inspected their VaR models which told them that risk levels were still low. Then they inspected their brains – and bailed out. Whether that was luck or judgement is still to be decided.
The great Victorian engineers built bridges that endure to this day because they couldn’t exactly model the risks. They built with a margin of safety not with a bonus on margins in mind. Remember this because the quants are not dead, they’re out there yet. They will rise again.
Related Articles: Risky Bankers Need Swiss Cheese, Not VaR, Hedge Funds Ate My Shorts, Black Swans, Tsunamis and Cardiac Arrests
I think most of these guys knew that the modelling was flaky. But who was going to tell the team leader, who would then have to tell the director to tell the board that they were going to stop doing something that was hugely profitable.
ReplyDeleteWho would want to be the CEO two years ago that had the job of telling shareholders that they made less money this quarter because of the possible risk to global financial markets in 2008.
Shareholders would have said "Carry on making money. We will sell out when you don't."
If you look over one-year time windows at volatility, you can see the transitions from uncorrelated (Gaussian) to correlated (Levy) statistics. Also, physicists (but not yet economists) have tools for modeling correlations at different scales. The renormalization group is probably the most famous. Correlations are also generally associated with broken symmetries in preferences.
ReplyDeleteGood article but it leaves me (as a quant-minded person) wondering why not just model that transition from uncorrelated to correlated.
ReplyDeleteThe last two sentences were great. "The quants are not dead... the will rise again." Remind me of a good old fashioned zombie movie. I can imagine the quants stiffly lumbering back from the graveyards coming to invest your money!
Combine this article with
ReplyDeletehttp://www.washingtonpost.com/wp-dyn/content/article/2009/07/05/AR2009070501587.html
and then you have the big analytical picture.
Graham
The point is though that economists don't run banks. To get to the top of a bank needs a single minded devotion to one's own career without consideration of others or the greater economy. Bank CEO's are not interested in macro-prudential regulation. Like Adam Smith's bakers all they want is to deliver rising profits to shareholders and build a bigger bank.
ReplyDeleteBut, of course, quant models showing politically-useful and grant-producingly scary projections of climate change are 'Settled science'!
ReplyDelete"they’ve brought the world to the point of financial meltdown thrice within a few years"...How can anyone write such simplifying insanity???
ReplyDeletefirst this is plainly wrong (SOME investment funds which says they follow a quant strategy - being true or not since it was fashionable to say it & increased AUM caused the summer 2007 event...the rest is a result of a general macro imbalances in which quant funds has been sucked in too), second it contradicts strongly the statements higher in the article that the world is too complex to model...how can one says this then give such over-simplifying statement about the causes of such crisis?? Looking for scapegoat has always been tempting in crisis times, but it does not solve anything...
& yes before you ask I am a "quant"...knowing that this is an umbrella term covering many many functions...from the basic s/sheet processing to complex modelling
I agree there has been over-complexification of models and that quants have been victim of their own success...while falling in love with their model they took years (sometimes) to build...but stating they have been responsible for almost meltdown as it is done in this article is simply a "eye-catching" headline that one would find in a free yellow press (for finance people)
Haven't been to this site before, but have to say that was the best non-quantitative explanation of the deficiencies of quants I've read so far. Plan to pass it on to some of my less hard-thinking friends. It's certainly a lot easier than Taleb's.
ReplyDeleteThanks.
I worked in Technology for JPMorgan in 2007 when the subprime mortgages (and interest-only, and other crazy schemes) were being described in mainstream media articles. Even as a Technology manager, having none of the training or experience of a Loan Officer, these approaches to lending seemed obviously ill-advised and faintly reprehensible -- how could they not prove harmful to the bank and to the borrower? I discussed these new mortgage offerings with colleagues and none of us understood how the bank could get involved in such transparently high-risk, and seemingly duplicitous arrangements. Perhaps we all lacked the depth of understanding of Risk Mgmt or Quantitative Analysis to accurately estimate the probability of loss, but we hardly needed them -- the taint of risk was that obvious. So the question this article begs that is truly interesting now, is what really went on at Goldman, that allowed them to exit before the house of cards collapsed? I'd like to think that somewhere (even at GS, perhaps especially at GS) the flame of logic did not gutter entirely, and that at least one Risk Mgmt team did successfully stand their ground and keep their firm out of harm's way. How GS actually did it may never come to light, but I would very much like to know, was it logic, or luck?
ReplyDeleteInteresting comments and I’d make just a couple of responses.
ReplyDeleteBlaming the quants themselves for the mess the world got into would be simply stupid – a bit like blaming scientists for discovering nuclear power. Lots of different interest groups were involved and a few minutes surfing the site, rather than just reading a single article, will show I’ve had a look at a number of them. Remember though that my aim here is to show, in a way most people can understand and without complex maths, how psychology and finance interact and, in this case, how the difficulties of modelling human behaviour contributed to this.
On which subject, the comment about modelling the switch from uncorrelated to correlated behaviour is an interesting one. There are lots of examples of this from physics – hysteresis in traffic flow and the transition from gaseous to solid states: critical points. Someone – Sornette? – did some modelling on this effect on markets around the turn of the century. I think they made some predictions which didn’t come true, but I’d need to go back and check.
I totally agree with the observation that GS's decision to take their own punch bowl away when they did is extraordinarily important to understand. If I ruled academia, it would also become required reading for anyone seeking an MBA, or managing other people's money...would love to have been a fly on the wall during that cat fight.
ReplyDeleteAs I understand GS insured it all with AIG. Uncle Sam bailing out AIG was what saved the day for GS rather than not playing the game. That's the payback for the letting the US Government nick all your previous CEO's, Rubin, Paulson, Corzine.
ReplyDelete"Back in 2006 when the CEO of Citigroup was still dancing in the last chance sub-prime disco the risk management team at Goldman Sachs got together and solemnly inspected their VaR models which told them that risk levels were still low. Then they inspected their brains – and bailed out. Whether that was luck or judgement is still to be decided."
ReplyDeleteQuibble: 2007 not 2006.
The quants by themselves are not so danngerous. But when you throw in incredible leverage and non-recourse sale of these financial derivatives it was only a matter of time till something exploded. All the pieces of the system are co-dependent: CEOs looking for short-term profits, boards without nerve to do the long-term thing, politically influenced regulators, simple-minded congressment trying to buy votes, rating agencies paid for performance, speculative simple investors unable or unwilling to actually read a prospectus, news media living on sound-bites not substance.
ReplyDeleteDid I miss anyone?
Keep the quants but cut the leverage and make the sellers retain an interest. That will get rid of most of the problems.