Singularities and Smoothness in Machine Learning

The business of Machine Learning is to predict the future. The above figure shows two examples. The left side figure is the monthly totals of international airline passengers, 1949 to 1960, and the right side figure is the yearly oil price evolution since 2000. Intuitively, it is much easier to predict the time series in the figure on the left than on the right.

On the left, the time series displays regular patterns and it has smooth ups and downs. This means that process has inertia which allows the predict the future. Instead, the right figure contains abrupt ups and down and there is no regularity.

We can see that any time series as the discretization of a real function. Hence we can that the above series correspond to the discretization of smooth and non-smooth functions.

In Math, there are levels of function smoothness. From the most smooth level to the worst level.

1. Analytic function (Taylor series converge)
2. Infinite differentiable function (Smooth function)
3. Finite differentiable function
4. Lipschitz continuous function (Piecewise differentiable)
5. Non-smooth function (Everywhere non-differentiable

Machine Learning while data comes from smoother functions much easier to make a prediction. The left time series would correspond to the second level: Infinite differentiable function since it looks predictable. The right time series corresponds to the fifth level non-smooth function. In this case, the function is not giving useful information to predict its future. This function is also known as a fractal function.

The analytic functions are the ideal setting to make predictions, it is sufficient to know a small period of time of an analytic function to predict its future and past at every moment as Laplace's demon does. If all data for Machine Learning came from analytical functions, all data scientists would be out of work.

When a function loses analyticity, a singularity occurs. A singularity is an event that breaks the present with its past, producing large errors in prediction. Singularities can have a local or global character. For example, the COVID-19 pandemic was a global singularity that disrupts all Machine Learning models around the world. Two years ago in Belgium, we had a local singularity on the weather forecast on a winter day. The forecasting service predicted that there will be no snow for the whole day, however we received the expected snow. The reason was the unexpected pollution produced by some industries in Germany that caused the snowfall.

In general, singularities in complex systems cannot be predicted accurately enough to perform anticipatory action. For example, I can predict that in the next 100 years I will be dead, but this prediction is useless.

Therefore, depending on how critical the prediction is, the predictive model must be complemented by a strategy to adapt the organization to a new business environment created by unexpected singularities. I finish, paraphrasing a phrase by Clarence Darrow mistakenly attributed to Darwin: "It is not the richest organization that survives, nor the one that uses Machine Learning the most, but the most adaptable to change."