Evaluate the problem properties
Evaluate the properties of the problem, such as dimensionality, linearity, smoothness, and convexity. Dimensionality refers to the number of variables or parameters involved in the problem. Higher-dimensional problems are more difficult and time-consuming to solve than lower-dimensional problems, and may require more sophisticated numerical methods, such as sparse matrices, grid methods, or dimension reduction techniques. Linearity refers to whether the problem can be expressed as a linear combination of some basis functions or not. Linear problems are easier and faster to solve than nonlinear problems, and can often be solved by direct methods such as matrix inversion, linear programming, or least squares. Nonlinear problems may require iterative methods, such as Newton's method, gradient descent, or quasi-Newton methods.
Smoothness refers to whether the problem has continuous and differentiable functions or not. Smooth problems are more stable and accurate than non-smooth problems, and can often be solved by analytical methods such as Taylor series, Lagrange interpolation, or spline interpolation. Non-smooth problems may require numerical methods that can handle discontinuities, singularities, or jumps, such as finite difference methods, finite element methods, or wavelet methods.
Convexity refers to whether the problem has a single global optimum or not. Convex problems are more tractable and reliable than non-convex problems, and can often be solved by global methods such as simplex method, interior point method, or branch and bound method. Non-convex problems may require local methods that can find suboptimal solutions, such as hill climbing, simulated annealing, or genetic algorithms.