New Interesting Resource for Machine Learning and Reinforcement Learning in Finance

Before introducing the new NYU class on 'Machine Learning and Reinforcement Learning in Finance' by Professor Igor Halperin, I briefly present my thoughts about the future intellectual challenges in this specific subject and field. 

The financial data is extremely noisy and is not stationary. The analysis of financial markets is challenging because of the presence of several factors (some of which are almost impossible to predict) that cause a various types of regime changes in the stochastic process. Designing a Machine Learning algorithm in Finance is more challenging [or at least very different] than it is for self-driving car and computer game (environments with fewer regime changes). However, this is not an obstacle. It is rather the opportunity to dig deeper.

1. The challenging environment of Finance requires going deeper in infinite hypothesis space and explore more candidate hypotheses with disciplined use of mathematical rigor, computational power, and model complexity.

2. The lack of 'big data' in Finance (at least in research) further complicates the task, and requires to stretch imagination for the feature engineering. However, the ultimate purpose is to successfully replace the features with hopefully insightful (not black box) algorithms.

3. Another promising direction is to synthesize the research of relevant signals coming from the fields that were either impossible to explore few years ago (i.e. sentiment analysis of market participants) or considered unrelated to financial analysis. 

4. I believe that it is possible to replace many static models with more accurate and up-to-date dynamic models based on already proven Machine Learning frameworks (i.e. triple loop of model selection, training [parameter search], and validation [hyper-parameter tuning] in supervised learning). Machine Learning methods can be used for gaining new insights about changes in correlations between traditional predictor and response variables. The ML models can systematically update predictive power, predicted intervals, and identity of independent variables. The dynamic model run 'online' can adjust importance of less relevant variables, or identify confounding variables defined at any level of granularity (i.e. principled use of PCA, Lasso, and several other methods).

5. Finally, I am most interested in the Reinforcement Learning paradigm. It requires a careful design of the reward, action-value, and state-value functions. Even though in Finance reward 'function' is relatively clear, we are in presence of extreme noise and a lack of stationary process.

Here, I would like to introduce the great new resource available in the form of online specialization by Professor Igor Halperin "Machine Learning and Reinforcement Learning in Finance" (please refer to the link below).

Professor Igor Halperin proposes three distinct Reinforcement Learning models. With the purpose of teaching, he went ahead and used discrete version of Black-Scholes-Merton's model (trees with three or more branches), as a 'base framework' - instead of conventional simple games.

This model can serve as a benchmark of the performance of diverse Reinforcement Learning algorithms beyond the financial field. Among other important concepts, Igor introduces his own research - which makes the class unique. He covers advanced topics of Reinforcement Learning and Inverse Reinforcement Learning applied to Finance, not yet implemented in practice. All codes are in Tensorflow but we all are welcome to rewrite or extend notebooks in our preferred frameworks.

Finally, I will also create a new GitHub repository for an open source research, where we will test and apply Machine Learning methods to financial data. Come & join the repository if that sounds interesting and challenging to you!

* link to specialization: New Specialization in Machine Learning in Finance offered by NYU

** link to the relevant paper by Professor Igor Halperin: Market Self-Learning of Signals, Impact and Optimal Trading: Invisible Hand Inference with Free Energy (Or, How We Learned to Stop Worrying and Love Bounded Rationality)

*** link to GitHub for the project: Projects on Reinforcement Learning in Finance