Existing economic models - Do they add value?

Economic models shape our everyday lives as they are the tools of decision makers in governments and central banks. Do these models have a successful track record and do they really work?? Applying them in real time and validating them on a day by basis, sends a clear message; they have serious shortcomings.?

The economic system is based on practices and business processes established at a time that predates modern technology. Governments and regulators try to make the best out of the system and manage the many challenges of the economy by thinking up new policy decisions and other measures to bolt on to our struggling banking and financial system. These measures are based on insights derived from outdated economic models formulated by economists to address the challenges of their respective eras. Rarely the question is asked, are these models still providing the right advice? Do they really work? My hands-on experience tells me otherwise; or at least the models are only applicable in very select circumstances and not appropriate for across the board policy decisions.

Real time predictive information service

In 1985, I founded Olsen & Associates as a research institute for applied economics with the goal of building a real-time information system using Big Data to provide banks and treasuries with decision-making support. The nascent computer technology made it possible to feed economic models with tick-by-tick market data to deliver real-time forecasts projecting minutes to hours, days, weeks and even months ahead, along with risk analytics and trading recommendations through the use of computer bots. We aimed to improve the decision-making quality of traders and managers in banks, asset management firms and corporate entities. We were inspired by the advances in natural sciences and in particular, new developments in meteorology. And had high expectations that a new generation of economic models could help such decision makers optimise their efforts and enhance their risk-adjusted performance

Our team of physicists and computer scientists started to build the real-time information system. However, we did not anticipate the many obstacles we would encounter and for the first two or three years, we always talked about our ‘three month’ project and did not appreciate the scale of our task.

Eventually, after many setbacks we managed to get our real-time information system off the ground and signed up major banks in Switzerland and Europe. At the time, leased lines were a necessity and the Internet did not yet exist, so there was no possibility of providing such a service to retail investors.??

First Conference of High Frequency Finance

Over the years, we published many scientific papers reporting our hands-on experiences and discoveries of statistical properties of financial markets that had not been known before. In 1995, we organised the first high-frequency finance conference. The event was attended by over 200 scientists from around the world and was a tremendous success. We showcased a data sample of foreign exchange market maker quotes with over 110 Mio market quotes. This large data sample, much bigger than what researchers had previously been able to access, was a wakeup call for the community about the challenges of analysing big data sets and the opportunities that high frequency data offered in terms of validating economic models and analysing their performance via tick-by-tick data. And how this enabled us to sidestep the problem of insufficient data that is so prevalent in economics.?

The conference rallied the community; the participants were full of hope that economics and finance would soon be transformed into a hard science built upon a solid foundation. The hope was that by deploying economic models in real-time and checking their performance against actual events on an ongoing basis we could help researchers identify shortcomings to eventually empower them to improve the quality of our economic models.?

In meteorology, forecasts were initially a laughing stock. Eventually, the models improved and became ever more powerful and effective. Our hope was to achieve the same with financial forecasting through the creation of strong predictive models and decision support tools.?

Unsatisfactory performance of economic models

Sadly, these hopes did not come true.

In 2001, we published? the book - ‘Introduction to high frequency finance’ by M. Dacorogna, R. Gen?ay, U. Müller, R. Olsen, O. Pictet at Academic Press. The book assembled under one cover all the research that we had conducted and published over the past 15 years, spanning a broad range of journals, many of them not easily accessible. The book provided an in-depth account of tick-by-tick market data, its statistical properties and how to build real-time forecasting and trading models. The book became a standard reference manual for the rapidly growing quant hedge fund communities that were developing algorithms to trade the markets.????

The performance of the model approaches described in the book were more sophisticated than the standard? Excel spreadsheet models using daily data, no doubt. But the performance improvement was only 20 to 30%. This was a far cry from the major breakthrough we had hoped for and nowhere near the successful breakthroughs that were occurring in other technologies that brought improvements of several orders of magnitude. Classical economics offered a good excuse: if there existed an opportunity to generate excess return, then this opportunity would anyway be arbitraged away and disappear, explaining the disappointing performance of any attempt to build better economic models.??

Over the years we had systematically explored literally every approach - from traditional fundamental models to time series models and technical analysis. You name it. We tried it.? Performance of all models was spurious.?

The best worked ‘a bit’; it was possible to achieve positive results, but only barely so; detailed performance reports are included in the book that we published.?

Today, we know that the existing economic models need hand holding and are not ready for 24/7 deployment. If we do, and run them 24/7 their performance is scrappy.? In natural sciences, models have to operate around the clock; this is taken for granted. How would it be otherwise possible to use airplanes, cars and computers? The fact of the matter is that the economic models used by policy makers do not work, when deployed real time. The inability of the models to explain movements in markets from minutes to hours, days, weeks and months, is a clear warning signal.? We argue that if they are not up to this task, then it is not permissible to use them for long term policy purposes.?

In natural sciences live testing is standard practice. How would it have otherwise been possible to develop the many gadgets and technologies that shape our modern lives? In our work, we have not shied away from the painful step of deploying models and tracking their performance. These many setbacks have been an opportunity to discover and evolve.? In the next post, I will explain the essence of our key discoveries and how the insights gained can be used to add value for the benefit of all from the everyday trader to regulators and policy makers.???