Real-time Forecasting
One of the problems for students of matters financial is that predicting stuff is very difficult, especially when it’s about the future. However, this pales into insignificance with the greater problem that we can’t even predict the present. In economic terms, basically, we really have no idea what’s happening in the world at any given time.
Predicting the present, snappily known as “nowcasting” is an area that economists spend lots of time worrying over, developing really neat algorithms that work right up to the next time they don’t. However, the rise of the internet has opened a new window on the world and analysts are now starting to nowcast by looking at trends emerging from data culled from the web. Perhaps, just perhaps, this can be turned this into a model which is near enough real-time to stand some chance of being useful.
Guess My Economics
Mostly we only find out what’s going on in global finance months or even years after the event. Stuff like whether we’re in a recession or not can only be figured out in retrospect, which isn’t much use if you’re tying to do useful economic things like fix interest rates or estimate demand for your products or produce investment analysis reports. In the latter case we use “useful” in the loosest possible sense, obviously.
This is the state of play when we’re dealing with major economic issues. When we’re dealing with less important ones commentators are usually forced to use their intuitions and years of experience. This is what us less knowledgeable people usually call “guessing” and is a significant problem, causing most economic and investment forecasts to be somewhat less than accurate. Basically, they’re what we call “crap”.
However, there is some hope for the experts. The rise of the worldwide web offers researchers a more or less real-time insight into the interests and concerns of the consumer at large. Once adjusted, by stripping out items of continual general interest such as “sex”, “porn”, “Google” (duh) and “interactive porn” (don’t ask) we find some interesting themes emerging. What’s more, they may even be useful.
Disease on the Net
Back in 2009 a bunch of researchers from Google and the Centers for Disease Control and Prevention published a paper that was both interesting and innovative, although they cunningly disguised this by using the title “Detecting Influenza Epidemics Using Search Engine Query Data”. The paper opened up a fascinating front on the use of the internet for nowcasting.
The researchers had hit on the idea of using Google search query frequency for epidemiological research. Basically, in our networked society, when people feel a bit under the weather they hit the search button before taking to their beds, and this allowed the researchers to track the spread of the disease by looking at flu related searches.
Unemployment on the Net
Inspired by this idea a series of economists have now set about trying to use the same idea for financial forecasting. In Google Econometrics and Unemployment Forecasting, Nikolaos Askitas and Klaus F. Zimmermann, make the point that this data may be particularly useful at the time at which it is most needed: in times of economic crisis when:
Demand on the Net
In yet another study, Hyunyoung Choi and Hal Varian, in Predicting the Present with Google Trends, investigated whether data culled from Google Trends was helpful in predicting more detailed supply and demand behaviour in consumers. Google Trends offers up query volumes in specific geographies for specific search items and the idea was that this data may provide evidence about what is happening right now, which will show up in future economic data:
Stilling the Waves
As we’ve seen before, in Twits, Butter and the Superbowl Effect, we need to be suitably sceptical about such findings: with enough data you can find something that plausibly correlates with pretty much anything as long as you have the processing power to go find it. But still, as far as economics goes this is pretty much real-time feedback and there are a bunch of other papers around from as far apart as Norway and Chile which seem to provide similar results.
Now, of course, these are early days and we don’t know what impact using this data will have on economic trends. All too frequently economics suffers from reflexivity, where cause and effect end up in a nasty feedback loop. Nevertheless having to hand a model that can provide fast and fairly accurate feedback in times of acute economic strife would revolutionise the tools central bankers and politicians have to manage such situations. Although, to be frank, as the current approach is like trying to hit a dartboard with a pin at twenty paces on the deck of ship in a Force 9 gale, that wouldn’t be too difficult.
As a supplement to economic forecasting tools, a way of helping to match supply and demand, it may also help reduce frictions in markets, moving us closer to the state of nirvana economists call “efficient markets”. Unlikely though this is, the approach looks like a genuinely useful supplement to existing econometric methods.
Related articles: Anatomy of a Growth Investor, James Randi and the Seer-Sucker Illusion, Twits, Butter and the Superbowl Effect
One of the problems for students of matters financial is that predicting stuff is very difficult, especially when it’s about the future. However, this pales into insignificance with the greater problem that we can’t even predict the present. In economic terms, basically, we really have no idea what’s happening in the world at any given time.
Predicting the present, snappily known as “nowcasting” is an area that economists spend lots of time worrying over, developing really neat algorithms that work right up to the next time they don’t. However, the rise of the internet has opened a new window on the world and analysts are now starting to nowcast by looking at trends emerging from data culled from the web. Perhaps, just perhaps, this can be turned this into a model which is near enough real-time to stand some chance of being useful.
Guess My Economics
Mostly we only find out what’s going on in global finance months or even years after the event. Stuff like whether we’re in a recession or not can only be figured out in retrospect, which isn’t much use if you’re tying to do useful economic things like fix interest rates or estimate demand for your products or produce investment analysis reports. In the latter case we use “useful” in the loosest possible sense, obviously.
This is the state of play when we’re dealing with major economic issues. When we’re dealing with less important ones commentators are usually forced to use their intuitions and years of experience. This is what us less knowledgeable people usually call “guessing” and is a significant problem, causing most economic and investment forecasts to be somewhat less than accurate. Basically, they’re what we call “crap”.
However, there is some hope for the experts. The rise of the worldwide web offers researchers a more or less real-time insight into the interests and concerns of the consumer at large. Once adjusted, by stripping out items of continual general interest such as “sex”, “porn”, “Google” (duh) and “interactive porn” (don’t ask) we find some interesting themes emerging. What’s more, they may even be useful.
Disease on the Net
Back in 2009 a bunch of researchers from Google and the Centers for Disease Control and Prevention published a paper that was both interesting and innovative, although they cunningly disguised this by using the title “Detecting Influenza Epidemics Using Search Engine Query Data”. The paper opened up a fascinating front on the use of the internet for nowcasting.
The researchers had hit on the idea of using Google search query frequency for epidemiological research. Basically, in our networked society, when people feel a bit under the weather they hit the search button before taking to their beds, and this allowed the researchers to track the spread of the disease by looking at flu related searches.
“Harnessing the collective intelligence of the millions of users, Google web search log can provide one of the most timely, broad reaching influenza monitoring systems available today. While traditional systems require 1-2 weeks to gather and process surveillance data, our estimates are current each day.”The raw trends are found here.
Unemployment on the Net
Inspired by this idea a series of economists have now set about trying to use the same idea for financial forecasting. In Google Econometrics and Unemployment Forecasting, Nikolaos Askitas and Klaus F. Zimmermann, make the point that this data may be particularly useful at the time at which it is most needed: in times of economic crisis when:
“Traditional flow of information is too slow to provide a proper basis for sound economic decisions”The trick in this study was to find relevant search items that correlated with the variable under study – German unemployment rates. Fascinatingly the study was not only able to use a small number of search queries to generate data predicting changes in employment rates ahead of the official figures, but broke down when a change in German economic policy, with the government supporting short-term working to get people back to work, started working.
Demand on the Net
In yet another study, Hyunyoung Choi and Hal Varian, in Predicting the Present with Google Trends, investigated whether data culled from Google Trends was helpful in predicting more detailed supply and demand behaviour in consumers. Google Trends offers up query volumes in specific geographies for specific search items and the idea was that this data may provide evidence about what is happening right now, which will show up in future economic data:
“We are not claiming that Google Trends data help predict the future. Rather we are claiming that Google Trends may help in predicting the present. For example, the volume of queries on a particular brand of automobile during the second week of June may be helpful in predicting the June sales report for that brand, when it is released in July”.And, indeed, when they used Google Trends data to supplement existing predictive models they got improved performance. In some cases startlingly improved performance: an 18% improvement in predicting “motor vehicles and part” sales and a 12% uplift in accuracy on “new housing starts” for instance.
Stilling the Waves
As we’ve seen before, in Twits, Butter and the Superbowl Effect, we need to be suitably sceptical about such findings: with enough data you can find something that plausibly correlates with pretty much anything as long as you have the processing power to go find it. But still, as far as economics goes this is pretty much real-time feedback and there are a bunch of other papers around from as far apart as Norway and Chile which seem to provide similar results.
Now, of course, these are early days and we don’t know what impact using this data will have on economic trends. All too frequently economics suffers from reflexivity, where cause and effect end up in a nasty feedback loop. Nevertheless having to hand a model that can provide fast and fairly accurate feedback in times of acute economic strife would revolutionise the tools central bankers and politicians have to manage such situations. Although, to be frank, as the current approach is like trying to hit a dartboard with a pin at twenty paces on the deck of ship in a Force 9 gale, that wouldn’t be too difficult.
As a supplement to economic forecasting tools, a way of helping to match supply and demand, it may also help reduce frictions in markets, moving us closer to the state of nirvana economists call “efficient markets”. Unlikely though this is, the approach looks like a genuinely useful supplement to existing econometric methods.
Related articles: Anatomy of a Growth Investor, James Randi and the Seer-Sucker Illusion, Twits, Butter and the Superbowl Effect
Check out MIT's "billion prices project" for nowcasting at its finest.
ReplyDelete