Earnings forecasting is a triumph of accountancy over reality:
given the complexity of most corporations an accurate forecast of earnings is logically,
and numerically impossible. Yet corporations
guide and analysts analyse and, mostly, they all end up looking wise.
It’s all nonsense, of course. The economy is a complex adaptive system
comprised of billions of working parts.
The realistic chance of anybody predicting anything minutely accurate
about anything of any interest is approximately zero, to several decimal
points. As any weather forecaster could
tell you.
Flat-bed Computers
Back in the 1960’s Edward Lorenz accidentally discovered something odd about weather forecasting. He needed to re-run a particular computer simulation of the weather and to save time decided to start it in the middle rather than at the beginning. Back in those days processing time was precious, “computers” weren’t things you carried around with you, unless you drove a flat bed truck, and user interfaces consisted of reels of punched tape or, if you were very lucky, punched cards (in my first job I saw people “debugging” cards by literally reading them and either patching holes with sticky tape or hand-punching new holes. Eat your hearts out, kids …).
Back in the 1960’s Edward Lorenz accidentally discovered something odd about weather forecasting. He needed to re-run a particular computer simulation of the weather and to save time decided to start it in the middle rather than at the beginning. Back in those days processing time was precious, “computers” weren’t things you carried around with you, unless you drove a flat bed truck, and user interfaces consisted of reels of punched tape or, if you were very lucky, punched cards (in my first job I saw people “debugging” cards by literally reading them and either patching holes with sticky tape or hand-punching new holes. Eat your hearts out, kids …).
As input for his new model run Lorenz used some of the numbers output from his previous simulation, taken from a printout of its results. To his
surprise the output of the second model run resulted in a completely different final
state of the program. Lorenz was dealing in theory and numbers, not actual weather, but the general thrust is that, instead
of predicting a dull day with sunny periods the computer program suggested that
Boston was going to obliterated by a freak super-cyclone. Or something like that.
Decimals
Now in general, if you give a computer program a particular set of input values you expect the same output. Software’s not renowned for setting off on its own personal journey of self-discovery, so this resulted in a certain amount of head scratching. Same input, completely different output? What the hell was going on?
Now in general, if you give a computer program a particular set of input values you expect the same output. Software’s not renowned for setting off on its own personal journey of self-discovery, so this resulted in a certain amount of head scratching. Same input, completely different output? What the hell was going on?
The answer was buried in the difference between the way the
computer crunched numbers and the way that the printout represented them. Internally the computer was using six decimal
places, but the output only showed three.
So when the computer generated (say) 0.506127 as an output what was
actually printed out was 0.506. What got
entered to restart the program was
0.506 and the computer interpreted this as 0.506000. What appeared to be happening
was that a difference of a few ten thousandths in the initial values was causing
the program to predict a completely different set of weather conditions.
This was shocking.
The standard view at the time was that small starting variations like
this would result in small variations of the outcome. Yet the evidence suggested otherwise. It was,
quite literally, chaos.
Nonlinear
Lorenz eventually wrote up his findings in one of those crucial papers that we dwell on from time to time: in Deterministic Nonperiodic Flow he noted that “slightly differing initial states can evolve into considerably different states”. More colloquially we know this as the butterfly effect, where the flap of a butterfly’s wings somewhere on the other side of the world can trigger major changes at home.
Lorenz eventually wrote up his findings in one of those crucial papers that we dwell on from time to time: in Deterministic Nonperiodic Flow he noted that “slightly differing initial states can evolve into considerably different states”. More colloquially we know this as the butterfly effect, where the flap of a butterfly’s wings somewhere on the other side of the world can trigger major changes at home.
Technically these types of system, typified by the weather,
are known as nonlinear, where the system output is not proportional to the input. Imagine Planet Earth going around the Sun, but occasionally deciding to visit Jupiter and Mars. In essence you can’t precisely predict the
outcome of any event in a nonlinear system.
This was a challenge to classical science – and classical economics. It turned out that prior to Einstein most scientists had spent most of their time studying the small subset of systems that do behave predictably, linear systems, which led Stanislaw Ulam to remark:
“Using a term like nonlinear science is like referring to the bulk of zoology as the study of non-elephant animals”.
The universe is largely made up of nonlinear systems – it’s
those that demonstrate linearity which are the unusual ones. Linear systems have a stable equilibrium
which, as equilibrium is the defining concept of standard economic theories,
brings us full circle to the problem that these theories don’t describe the
real world. The economy is an example of a specific class of nonlinear system, known as complex adaptive systems, which continually adapt to changes in their environment.
Non-Equilibrium
Complex adapative systems can appear to be stable, in equilibrium, for long periods, because of this inherent ability to modify themselves in response to changing situations. In fact they’re not, and are maintaining an apparent equilibrium by dint of exchanging energy with other systems: in our case the Sun or fossil fuels would be a favored source. In the investment markets the equilibrium we often perceive is actually an uneasy standoff between powerful opposing forces, such as value and sentiment driven investors.
Non-Equilibrium
Complex adapative systems can appear to be stable, in equilibrium, for long periods, because of this inherent ability to modify themselves in response to changing situations. In fact they’re not, and are maintaining an apparent equilibrium by dint of exchanging energy with other systems: in our case the Sun or fossil fuels would be a favored source. In the investment markets the equilibrium we often perceive is actually an uneasy standoff between powerful opposing forces, such as value and sentiment driven investors.
Because the economy often demonstrates stability over relatively
long periods people tend to discount bad stuff that’s happened in the past: the behavioral issue generally described as disaster myopia. Eventually
everyone forgets that bad stuff is possible – and as soon as that happens you
find powerful forces in the markets agitating for the removal of “unnecessary”
regulations that were originally introduced in the wake of some disaster or
other.
In one of those peculiarly inverted arguments that heavily incentivised people are
often prone to, the fact that the disaster that regulation was introduced to
prevent hasn’t happened is presented as proof that the regulation isn’t needed. Unfortunately those damn butterflies keep
flapping, and as soon as the balance of equilibrium is moved even a tiny bit it
only takes one particularly vigorous specimen to send the whole kit and
caboodle into a tail-spin.
Analysts' Earnings Forecasts
Analysts' Earnings Forecasts
In the midst of all this we have the modern day oracles and
soothsayers doing their best to appear knowledgeable by predicting the earnings
of corporations. In general they’re not
very good at it: research by Lawrance Brown in Interactions Between Analyst Earnings Forecasts and Management Earnings Forecasts shows that analysts
introduce errors, and the more they introduce errors the more corporations have
to issue guidance in order to correct them.
The reason that managements may be better at forecasting
than analysts is that CEOs seem to be judged on this by their boards. As Lee, Matsunaga and Park relate in
Management Forecast Accuracy and CEO Turnover:
“Our results suggest that the labor market penalizes managers for inaccurate forecasts and that managers with low ability bear a cost from issuing inaccurate forecasts. In addition, our study suggests that the board holds the CEO responsible for the accuracy of the firm’s earnings forecasts.”
Corporations' Earnings Forecasts
However, the evidence that corporations go to great lengths to meet earnings forecasts is significant. The nonlinear world is just too complex to permit this, so when corporations hit their earnings forecasts “spot-on” investors should take this as a warning sign and start looking for evidence of earnings manipulation. Dawn Matsumoto describes the results of her research on this:
However, the evidence that corporations go to great lengths to meet earnings forecasts is significant. The nonlinear world is just too complex to permit this, so when corporations hit their earnings forecasts “spot-on” investors should take this as a warning sign and start looking for evidence of earnings manipulation. Dawn Matsumoto describes the results of her research on this:
“I also examine whether firms manage earnings upward or guide analysts' forecasts downward to avoid missing expectations at the earnings announcement. I examine the relation between firm characteristics and the probability (conditional on meeting analysts' expectations) of having (1) positive abnormal accruals, and (2) forecasts that are lower than expected (using a model of prior earnings changes). Overall, the results suggest that both mechanisms play a role in avoiding negative earnings surprises”
So, basically, some corporates artificially massage their numbers upwards, using accruals, and some downwards, in order to meet earnings forecasts. And, increasingly, investors are treating overly accurate profits with the suspicion that such results deserve. Which is potentially unfortunate for those corporations whose results are genuine, but which is a classic result of situations involving asymmetry of information between buyers and sellers (see: Akerlof's Lemons: Risk Asymmetry Dangers for Investors).
Spurious Accuracy
In a nonlinear world comprised of vicious butterflies and sundry other dangerous insects this is the right approach. The only reason that corporations can be relatively sure of hitting earnings targets at any reasonable time horizon is because they can control and manipulate the numbers. A little smoothing is fine, but if this morphs into wholesale manipulation we need to beware. What starts innocently can too often lead to the worst kind of destructive capitalism.
Spurious Accuracy
In a nonlinear world comprised of vicious butterflies and sundry other dangerous insects this is the right approach. The only reason that corporations can be relatively sure of hitting earnings targets at any reasonable time horizon is because they can control and manipulate the numbers. A little smoothing is fine, but if this morphs into wholesale manipulation we need to beware. What starts innocently can too often lead to the worst kind of destructive capitalism.
Lorenz's conclusion was that the weather could not be predicted with any great accuracy. The same is true of the stockmarket in general and most corporations' results in particular. The world is too complex, too adapative and too full of flapping butterfly brained people for it to ever be otherwise.
While I do agree that on an individual basis, it's near impossible for an individual analyst to accurate predict the fundamentals of a company 4-8 quarters into the future, there are two things I'd like to bring up.
ReplyDeleteOne, the research which has been done to date has focused exclusively on the narrow data set of Sell Side estimates. This group has a very flawed incentive structure which leads to a skewed set of estimates. It does not include buy side and independent analysts who are not shackled by that same incentive structure, they are free to share unbiased analysis which has been proven to be more accurate (more on that in a second).
And two, you aren't bringing into the picture the wisdom of the crowds here. Earnings forecasts are not about what one individual analyst thinks. They are about what the group, the market as a whole, thinks. Again, the current sell side data set is flawed. But when you open it up to a larger set of individuals, the wisdom of the crowds is able to take effect and you get a far more accurate set of predictions.
This is why we built Estimize, to open this up to the whole financial community. It turns out that when there are 20+ estimates for an earnings release, 77% of the time the Estimize consensus is more accurate than the sell side consensus.
So yes, company management can move the numbers, but over time as the crowd grows larger and more diverse, you're going to see consensus earnings models from Estimize which are scary accurate. We're already seeing it.
Hi Leigh
ReplyDeleteYou're absolutely correct that focusing on sell-side analysis may give a skewed picture of the general accuracy of the prediction market. The research on buy-side analysis is very light, but what there is indicates that there isn't much difference in the outcome - see Do buy-side analysts outperform the sell-side. The authors have published a number of papers in the same vein providing a similar answer: no, they don't, although it's not entirely clear why this is so.
As to the wisdom of crowds, I've written about this a number of times - see, for instance The Wisdom of Internet Crowds or Noise, Sentiment and StockTwits. It's an interesting topic and I'll be fascinated to see if such techniques are robust against behavioral biases as they become more popular.
Thought provoking stuff as usual. :) The sore thumb for me though was the paragraph about regulation.
ReplyDeleteSurely introducing regulation is a pretty big butterfly flap into the economic system? I think it's pretty clear regulation distorts systems, as we're seeing happen in real time with banking, now. For instance, banks are deleveraging and boosting capital ratios -- and so become less risky -- but the predictable consequence is that risk is being pushed up to the state level,. We're even seeing more and more pundits call for direct lending from the state to businesses, home owners, and so forth.
The blow-up that would eventually result would be a different kind of blow up, and we'll never know what lower or different state spending might have done for good or ill.
There's no easy answer to this -- I favour massive and collective first-aid reserves, partnership structures, and living wills rather than excessive regulation, but all of these present their own problems and distortions, too. (Note least where do you keep 20 years worth of set-aside payments from banks in between crashes, and how do you stop that wall of money distorting things, too!? :) )
I favour massive and collective first-aid reserves, partnership structures, and living wills rather than excessive regulation, but all of these present their own problems and distortions, too.
ReplyDeleteAnd none of which can be introduced without legislation and further regulation ... But my point really was about the contribution that deregulation had on the banking crisis. There are some industries, of which banking is the prime example, where relying on competition to regulate behavior has failed many times. In particular the idea that disclosure is enough to allow customers to self-regulate financial markets is a busted flush.
The best time to introduce this type of regulation is when the system is already broken. The problem is persuading people to stick to it when the latest crisis becomes a distant memory.
True, to an extent. I guess I'd distinguish between regulation that sets the rules, and regulatory oversight.
ReplyDeleteI basically do not believe that any regulators can look into massive financial organisations and ensure they don't blow up -- at least not without a truly spectacular simplification of the financial system, that would take us back to the 19th Century, and have a big knock-on effect on trade.
Worse, I think the longer they seemed to do so successfully, the bigger the eventual blow-up. Minksy moments and all that.
Anyway, I certainly agree stepping back to the halfway house of say the early 1980s regulation wise would help.
And I'm not friend of financial insiders, that's for sure. If I was going to introduce draconian, unworkable legislation, the first thing I'd introduce is a cap on banking salaries and bonuses of £1 million.
(I know, not realistic. But we're dreaming, right? ;) )
Timarr,
ReplyDeleteI've definitely read that buy side paper, but here's the difference. That paper is talking about stock recommendations, which are a fools game to begin with. "Stock Recommendations" are something the sell side does as a gimmick, it is a hold over from the brokerage days. The buy side does not engage in this type of activity in any meaningful way. We don't recommend stocks buy/sell/hold, that's ridiculous.
Estimize is about crowdsourcing earnings expectations, now stock recommendations. There are a few academics at Chicago Booth who are doing research on our data set right now. While we've done some analysis ourselves, I'm interested to see what they find as well.
I basically do not believe that any regulators can look into massive financial organisations and ensure they don't blow up -- at least not without a truly spectacular simplification of the financial system, that would take us back to the 19th Century, and have a big knock-on effect on trade.
ReplyDeleteWell, here's one possible answer: The End of Finance, As We Know It"
:)))
I've definitely read that buy side paper, but here's the difference. That paper is talking about stock recommendations, which are a fools game to begin with. "Stock Recommendations" are something the sell side does as a gimmick, it is a hold over from the brokerage days.
ReplyDeleteThe same researchers have looked at earnings forecasts, but the paper isn't freely available on the web: Buy-Side vs. Sell-Side Analysts' Earnings Forecasts. The abstract gives a flavor:
The tests show that the buy-side analysts made more optimistic and less accurate forecasts than their counterparts on the sell side. The performance differences appear to be partially explained by the buy-side firm's greater retention of poorly performing analysts and by differences in the performance benchmarks used to evaluate buy-side and sell-side analysts.
But if there's any new research or evidence point me at it when it comes out and I'll write it up. All in favor of people using new technology to break up the old models.