Self-Organising Wardrobes
In the long-term we’ll all be dead because, in the long-term, there is no escape from the iron hand of thermodynamics which tells us that every system moves from a state of order to one of disorder. It’s a bit like your wardrobe spontaneously re-organising itself. Only when it’s finished you can’t actually find anything in it any more. Neat but bloody useless.
The measurement of order in a system is known as entropy and the idea that entropy always increases is bound up with the idea that energy ultimately moves from a useful and usable state to one in which it’s unusable and useless. And this is as true of stockmarkets as any other system – not even the perfect demon can outperform the markets other than by luck unless they can also escape the laws of physics. Entropy rules – OK?
Maxwell’s Demon
Back in the nineteenth century the Scottish scientist James Maxwell came up with a thought experiment that he believed might invalidate this entropic principle, the Second Law of Thermodynamics. He postulated a box divided into two with a door in the partition. The box starts with molecules in both halves, zipping around randomly. Maxwell then invented the concept of an omniscient demon, capable of knowing the speed of the molecules, who opened the door to let through the faster ones whenever they approached – but from one side only.
This, Maxwell believed, would invalidate the Second Law – because the system would become increasingly ordered, as fast molecules and slow ones get separated, and therefore lose entropy. Thermodynamics doesn't say that this can't happen, because it's based on statistical probability, but it does rule out the possibility of it happening in a non-random way. And the demon is decidedly non-random.
As with many philosophical experiments this rather ignored the idea that an omniscient demon doesn’t actually exist on the grounds that there’s no known reason why one couldn’t. This type of speculation is typical of metaphysics, where philosophers invent theoretically possible things which don’t actually exist to prove a point. Bizarre, but true.
Information Rules
It took a while for people to figure out what was wrong with this argument because it really isn’t obvious. The problem is not the existence or otherwise of an impossible being but the impossibility of the impossible being actually knowing the speed of the molecules. The difficulty lies in measurement and the root of the problem is our old friend, entropy. There's a good overview of the history in this paper by John Earman and John D. Norton.
At issue is the problem that in order to make a decision to open or close the door the demon needs to figure out whether an approaching molecule is a fast one or not. Based on our knowledge of physics this requires processing effort on behalf of the demon and, if you run the numbers, it turns out that it’s going to have to expend more energy – and therefore increase entropy – than will be created by letting through the faster molecules. And thus the Second Law of Thermodynamics will be preserved.
Although this, of course, assumes that the demon is governed by the laws of physics – which is precisely the assumption that Maxwell had to make. Philosophers can invent impossible creatures to use in their thought experiments but these beings aren’t allowed to violate the fundamental laws of the cosmos.
Demon Investing
Anyway, a hundred and fifty years on Maxwell’s demon has gotten bored manipulating molecules and in its spare time has discovered that Louis Bachelier proved that stocks behave in the same apparently random way that molecules do. Being smart the demon figures out that this is an opportunity to use its molecule manipulating methods to make some quick money.
So now instead of a container enclosing gas molecules the demon is in the world of the stock market, trying to figure out which are the high growth stocks it wants to let into its portfolio and which are the low growth ones it wants to keep out. Occasionally it opens a door to let one through or, as we put it in the investment world, “buys a stock”. It could even “sell a stock” occasionally, letting stocks out of the door instead of in.
With perfect knowledge at all times of all the stocks in the system, the demon can always identify the correct stocks to let in and those to let out. This, of course, more or less describes how the efficient markets hypothesis sees individual investors, making decisions on perfect information. In accordance with the Second Law of Thermodynamics, then, such a being is impossible: to assimilate all of the information needed to identify the correct stocks would reverse the direction of entropy.
Demons Everywhere
To approximate closer to the real-world we actually don’t have one demon, we have millions, all opening and closing their little portfolio doors frantically. So our particular demon investor, as postulated above, is actually working on the basis of pure fundamentals: it’s identifying those stocks which are underpriced or overpriced. However, in this demon driven world, all the other demons are doing exactly the same thing. So in fact we once again move back to efficient markets, because as soon as any stock moves from its equilibrium position then all the demons will immediately trade it one way or another. Once again the Second Law is violated.
In reality every time each demon opens their door they introduce friction into the equation: there’s a cost to every trade which means that money – or energy – leaves the system. It’s not closed. In fact it’s completely open to billions of dollars flowing from behaviourally challenged demons to the securities industry. In fact to acquire information will usually also require the expenditure of money – to buy newspapers, access real-time information or simply use time which could be used to make money in other ways. After all, presumably someone pays the demon to do its day-job of molecule management.
Demon Satisficing
In practice our demon isn’t going to be able to track every single molecule or stock. It’s going to take short-cuts. So when a molecule – or stock – comes to its attention it’ll likely analyse it and make a decision about whether to admit it or not based on its own qualities in relation to the small subset of others that the demon has information about. This looks more like the satisficing algorithms of Herbert Simon than anything else we’ve seen. The question of how a stock comes to the demons’ attention is another big question, which we’ll leave for now.
So we can easily imagine a particularly dumb demon opening and closing its little door at random and accidentally acquiring a great portfolio, which they then attribute to skill. This doesn't violate the Second Law because it's just randomness in action; after all the demon isn't actually expending much energy in making its decisions.
Or if we heat up the entire container – and see the stock market rise – then we’ll likely a correlation between the stocks in the portfolio and those on the outside: a rising tide floats all boats. We can also imagine serried ranks of demons figuring out they’re performing worse than the market, propping open their doors permanently and buying all the stocks. Index demons, indeed.
Retiring the Demon
It’s important not to take the analogy too far. After all, multiple demons can have the same stock in their portfolio, while it’s not physically possible for them to share the same molecule: although there are bizarre phenomena at the microscopic level which might allow that to be overcome: the Second Law may not hold at the quantum level, either, but that’s another story.
Although the analogies are by no means perfect Maxwell’s Demon Investor gives us an insight into why it’s very difficult for individual investors to defeat the market averages. Of course, because each investor doesn’t sit in their own thermodynamically sealed box it’s possible for the individual to outperform but thermodynamics and stock movements are generally about statistics: to defeat the statistical averages and overcome the frictional costs of information acquisition an individual or an institution needs to be better at converting low-entropy information into stock choices than the others around them.
Meanwhile Maxwell’s Demon has retired and gone back to molecule flipping full-time. It’s a lot easier than making a profit by picking stocks.
Related articles: O Investor What Art Thou Rational?, Econophysics, Consciousness and Cosmic Karma, Satisficing Stockpicking
In the long-term we’ll all be dead because, in the long-term, there is no escape from the iron hand of thermodynamics which tells us that every system moves from a state of order to one of disorder. It’s a bit like your wardrobe spontaneously re-organising itself. Only when it’s finished you can’t actually find anything in it any more. Neat but bloody useless.
The measurement of order in a system is known as entropy and the idea that entropy always increases is bound up with the idea that energy ultimately moves from a useful and usable state to one in which it’s unusable and useless. And this is as true of stockmarkets as any other system – not even the perfect demon can outperform the markets other than by luck unless they can also escape the laws of physics. Entropy rules – OK?
Maxwell’s Demon
Back in the nineteenth century the Scottish scientist James Maxwell came up with a thought experiment that he believed might invalidate this entropic principle, the Second Law of Thermodynamics. He postulated a box divided into two with a door in the partition. The box starts with molecules in both halves, zipping around randomly. Maxwell then invented the concept of an omniscient demon, capable of knowing the speed of the molecules, who opened the door to let through the faster ones whenever they approached – but from one side only.
This, Maxwell believed, would invalidate the Second Law – because the system would become increasingly ordered, as fast molecules and slow ones get separated, and therefore lose entropy. Thermodynamics doesn't say that this can't happen, because it's based on statistical probability, but it does rule out the possibility of it happening in a non-random way. And the demon is decidedly non-random.
As with many philosophical experiments this rather ignored the idea that an omniscient demon doesn’t actually exist on the grounds that there’s no known reason why one couldn’t. This type of speculation is typical of metaphysics, where philosophers invent theoretically possible things which don’t actually exist to prove a point. Bizarre, but true.
Information Rules
It took a while for people to figure out what was wrong with this argument because it really isn’t obvious. The problem is not the existence or otherwise of an impossible being but the impossibility of the impossible being actually knowing the speed of the molecules. The difficulty lies in measurement and the root of the problem is our old friend, entropy. There's a good overview of the history in this paper by John Earman and John D. Norton.
At issue is the problem that in order to make a decision to open or close the door the demon needs to figure out whether an approaching molecule is a fast one or not. Based on our knowledge of physics this requires processing effort on behalf of the demon and, if you run the numbers, it turns out that it’s going to have to expend more energy – and therefore increase entropy – than will be created by letting through the faster molecules. And thus the Second Law of Thermodynamics will be preserved.
Although this, of course, assumes that the demon is governed by the laws of physics – which is precisely the assumption that Maxwell had to make. Philosophers can invent impossible creatures to use in their thought experiments but these beings aren’t allowed to violate the fundamental laws of the cosmos.
Demon Investing
Anyway, a hundred and fifty years on Maxwell’s demon has gotten bored manipulating molecules and in its spare time has discovered that Louis Bachelier proved that stocks behave in the same apparently random way that molecules do. Being smart the demon figures out that this is an opportunity to use its molecule manipulating methods to make some quick money.
So now instead of a container enclosing gas molecules the demon is in the world of the stock market, trying to figure out which are the high growth stocks it wants to let into its portfolio and which are the low growth ones it wants to keep out. Occasionally it opens a door to let one through or, as we put it in the investment world, “buys a stock”. It could even “sell a stock” occasionally, letting stocks out of the door instead of in.
With perfect knowledge at all times of all the stocks in the system, the demon can always identify the correct stocks to let in and those to let out. This, of course, more or less describes how the efficient markets hypothesis sees individual investors, making decisions on perfect information. In accordance with the Second Law of Thermodynamics, then, such a being is impossible: to assimilate all of the information needed to identify the correct stocks would reverse the direction of entropy.
Demons Everywhere
To approximate closer to the real-world we actually don’t have one demon, we have millions, all opening and closing their little portfolio doors frantically. So our particular demon investor, as postulated above, is actually working on the basis of pure fundamentals: it’s identifying those stocks which are underpriced or overpriced. However, in this demon driven world, all the other demons are doing exactly the same thing. So in fact we once again move back to efficient markets, because as soon as any stock moves from its equilibrium position then all the demons will immediately trade it one way or another. Once again the Second Law is violated.
In reality every time each demon opens their door they introduce friction into the equation: there’s a cost to every trade which means that money – or energy – leaves the system. It’s not closed. In fact it’s completely open to billions of dollars flowing from behaviourally challenged demons to the securities industry. In fact to acquire information will usually also require the expenditure of money – to buy newspapers, access real-time information or simply use time which could be used to make money in other ways. After all, presumably someone pays the demon to do its day-job of molecule management.
Demon Satisficing
In practice our demon isn’t going to be able to track every single molecule or stock. It’s going to take short-cuts. So when a molecule – or stock – comes to its attention it’ll likely analyse it and make a decision about whether to admit it or not based on its own qualities in relation to the small subset of others that the demon has information about. This looks more like the satisficing algorithms of Herbert Simon than anything else we’ve seen. The question of how a stock comes to the demons’ attention is another big question, which we’ll leave for now.
So we can easily imagine a particularly dumb demon opening and closing its little door at random and accidentally acquiring a great portfolio, which they then attribute to skill. This doesn't violate the Second Law because it's just randomness in action; after all the demon isn't actually expending much energy in making its decisions.
Or if we heat up the entire container – and see the stock market rise – then we’ll likely a correlation between the stocks in the portfolio and those on the outside: a rising tide floats all boats. We can also imagine serried ranks of demons figuring out they’re performing worse than the market, propping open their doors permanently and buying all the stocks. Index demons, indeed.
Retiring the Demon
It’s important not to take the analogy too far. After all, multiple demons can have the same stock in their portfolio, while it’s not physically possible for them to share the same molecule: although there are bizarre phenomena at the microscopic level which might allow that to be overcome: the Second Law may not hold at the quantum level, either, but that’s another story.
Although the analogies are by no means perfect Maxwell’s Demon Investor gives us an insight into why it’s very difficult for individual investors to defeat the market averages. Of course, because each investor doesn’t sit in their own thermodynamically sealed box it’s possible for the individual to outperform but thermodynamics and stock movements are generally about statistics: to defeat the statistical averages and overcome the frictional costs of information acquisition an individual or an institution needs to be better at converting low-entropy information into stock choices than the others around them.
Meanwhile Maxwell’s Demon has retired and gone back to molecule flipping full-time. It’s a lot easier than making a profit by picking stocks.
Related articles: O Investor What Art Thou Rational?, Econophysics, Consciousness and Cosmic Karma, Satisficing Stockpicking
An interesting article as ever, thank you. I found the comparison of net loss of energy from the system through friction effects to the effect of fees&other costs of investing a particulalry useful analogy.
ReplyDeleteOne correction on the science though; in thermodynamics the universe tends toward disorder not towards order. Spontaneous events can only occur if net entropy (that is net disorder) increases. The solid (ordered) ice cube melts in a drink, etc. Thermodynamics does allow for localised decreases in entropy (i.e. increases in order) to occur in a system - the a/c unit cools the room, thus increasing the order of the air molecules; they move more slowly - but only in return for a greater increase of entropy outside the system - the a/c unit warms the air outside the room to yield net increase in entropy. (as an aside in thermodynamic terms that's also why body temperature is higher than air temperature in most climates; and why we sweat in hot climates).
Hi John
ReplyDeleteYou are - of course - correct. Entropy is, if anything, a measure of disorder not order. In fact these days it's generally not linked to order at all, for various reasons too complex to deal with here, and it's probably more correct to define entropy in terms of dispersal of energy.
Anyway, I need to go correct my cracked text :) Thanks!