Mysticism in Finance! In the 21st Century?

History often fascinates, but it can be hard to empathise with past societies that today we might have expected to be rational - and weren't. The Salem witch purge was illogical, and, predicting the weather by a examining a sheep's entrails just, well, didn't (and still doesn't) work.

Mysticism still exists though, of course, in widely acknowledged forms. Astrology is the major example; horoscopes, tarot cards and psychic readings. Its not my thing (I'm rather entrenched in the science camp), but I'm not the Richard Dawkins type either, I don't actively want to question people's faith in these things.

But I do want to question people's faith in a financial domain. Its interesting chatting with so-called rational, non-believers who repeatedly and publicly, but completely inadvertently, bring mysticism to their daily jobs. If you work in finance, or even if you don't, I think it very likely you will have come across someone forecasting a stock or a futures price, like an old haruspex forecast the weather: "it's bounced off the 38.2% fibonnaci retracement and formed a morning star pattern... and therefore its going up!" Marvellous, get me in, it will be a bountiful harvest. Except that 50% of the time it won't be!

Because you see, its difficult, even for scientists, to apply a reliable scientific method to verify any particular conclusion. And furthermore its difficult to reliably confirm that that scientific method is itself reliable. This is why, so often, the right questions, or cross-validations, are not asked of the forecaster. Combined with optimism and a desire for the forecast to hold true there just aren't necessarily the right barriers to decision making.

In my opinion, the best decision makers and best analysts (and there are lots of them, some of whom I have had the pleasure of working with) know precisely what data they have collected. They know precisely what data they have excluded. They indirectly account for the unknown data they don't know they are missing. They know the tools and the techniques for cross-validating their models and they keep track of when things worked and when things failed. Failures are often the most useful information of all, and drive the evolution of new creative ideas. Lastly they have good instinct, some unexplainable ability to piece everything together and consistently do well. This I suppose is the tricky part, because its not completely indistinguishable from some form mysticism.

But there is a difference and it might not actually be that difficult to observe. Someone who bases a conclusion on simple patterns without any form of statistical tests compared with someone assimilating factual and well analysed data, really had better hope Mars aligns with Venus. My Mum took the wind readings today, she looked at the satellite images, modelled the air pressure changes and said its not going to rain. My Dad said it rained yesterday so its going to rain again today. Will we all be needing umbrellas?