How can nonlinearity be accounted for in optimization models?

The intangible example of nonlinearity is the word platforms. The positive network effect improves the quality of user experience by increasing the number of users of a platform. A good loop of feedback causes for example Facebook from 2 users in a few years reach to 2 billion users! Remember the Metcalfe’s law.

Any book on nonl linear programming will help you to underatnd non linear optimization If your function is convex, AMPL has a feature that allows you to use piecewise linear function replacing a linaer function. I and one of my former doctoral students have used this feature. in case you are not sure exact nature of non linerarity, you can use ANN

Why is it non linearity important? Maybe because (almost) all processes are... non linear in essence? True, we approximate them, often with great results, with linear tools but most processes that we try to model are intrinsically non linear. That said and as mentioned in the article, non linearity comes with a cost. Not only is it complex to model non linearity but we don't have much tools to do so and they are certainly not as powerful as the "simple" linear tools that are available today. Sometimes we also can piece wise approximate the non linearity when we know where to look at, i.e. at existing solutions. Sometimes, we have to embrace non linearity and use non linear models and tools. This is a rather technical subject.

Identifying nonlinearity in data science involves strategic exploration. Visualize relationships through feature and residual plots, uncovering unexpected patterns. Use advanced analytics like polynomial features and decision trees to capture complexities. Statistical techniques, such as nonlinear correlation metrics and kernel methods, offer deeper insights. In a recent project predicting customer engagement, nonlinear patterns emerged post-polynomial transformations, significantly enhancing model accuracy. Embracing nonlinearity is key in optimizing data-driven insights, and ensuring robust predictive models.

My first recommendation is that if a nonlinear optimization model can be tackled efficiently in terms of time and global optimality conditions, just move forward with it. If not, then try to transform this nonlinear model into a linear one. There a few ways to do that. If there is no way to transform, the next step may be decomposing the model and solving the non-linear part with a machine learning (ML) model (i.e. using a deep learning with ReLu functions) and integration the ML model in to the linear model.

The problem is not to identify non linearity in a model... The problem is how to detect non linearity and how to approximate non linearity with linearity in a meaningful way. First comes the problem, then come the models... not the other way around.

Use non linear methods, models and solvers! I would like to emphasis the fact that if you use second order methods, you need to approximate the Hessian depending on your problems and contexts while with first order methods, you have a very generic method that can be used out of the box. Such is the difficulty of non linearity!

Identify the trend, ignore the noise. To solve nonlinear models, my go-to method is to: 1.) analyze the trend (assume a linear trend or a polynomial trend) 2.) fit a linear regression model to the data 3.) extract the detrended values. As the name suggests, the detrended values should not have any observable trend (linear or nonlinear) in our output values from which the trend is subtracted. If we still see trends in this data, use a higher degree curve, rinse and repeat. One must be careful in deciding the degree of the curve to avoid overfitting which ultimately is capturing the noise in the model. Using RMSE score and the R2 value can help optimize the fit of the curve if using this methodology

Before you resort to using methods for nonlinear models, think about linearization. In some models only a single constraint might be nonlinear. In that case it might be more efficient to try linear approximation assuming the final feasible region is still a convex polytope.

"Optimization is not about Linearity versus Nonlinearity; it is about Convexity versus Nonconvexity." - Terry Rockafellar Indeed, categorizing optimization problems solely based on the distinction between nonlinear and linear is not always a reliable criterion. There exist nonlinear optimization problems that can be tackled more efficiently than certain linear ones. The effectiveness of optimization solutions often depends on various factors beyond the simplicity or complexity associated with linearity or nonlinearity.

Statistically nothing is linear, and mathematically almost everything (after lot of transformations). Normal human nature of thinking, taking decisions, suggesting - all are in a linear way. However, in reality it is just assumption. Even in linear regression problem with a certain use-case, we can’t have the best fit universal line, ever!