What are some challenges and solutions for solving stiff ordinary differential equations numerically?

1 Stability and stiffness

One of the main difficulties with stiff ODEs is that they can cause numerical instability, meaning that small errors in the approximation can grow exponentially and lead to wrong or divergent solutions. This can happen even if the exact solution is stable and well-behaved. To avoid instability, you need to choose a suitable numerical method and a sufficiently small step size. However, this can also make the computation very slow and costly. A measure of stiffness is the ratio of the largest to the smallest eigenvalue of the Jacobian matrix of the ODE system. The larger the ratio, the stiffer the system, and the more restrictive the stability conditions.

2 Implicit and explicit methods

A common way to classify numerical methods for ODEs is whether they are implicit or explicit. Explicit methods use only the current and previous values of the variables to compute the next value, while implicit methods use also the future value, which requires solving a nonlinear equation or system at each step. Explicit methods are usually simpler and faster, but they have more stringent stability requirements, especially for stiff ODEs. Implicit methods are more stable and can handle larger step sizes, but they are more complex and slower, and may need iterative solvers or matrix factorizations.

3 A-stability and L-stability

A useful property for numerical methods for stiff ODEs is A-stability, which means that the method is stable for any step size, regardless of the stiffness of the system. A-stable methods can avoid instability and allow for larger step sizes, but they may not be accurate or efficient enough for some problems. Another property that is desirable for stiff ODEs is L-stability, which means that the method also dampens the high-frequency components of the solution, which are often irrelevant or noisy. L-stable methods can reduce oscillations and errors, but they may also introduce artificial damping or phase lag.

4 Backward differentiation formulas

One of the most widely used classes of methods for stiff ODEs is backward differentiation formulas (BDFs), which are implicit multistep methods that use backward differences to approximate the derivatives. BDFs have several advantages, such as high order of accuracy, A-stability, and L-stability. They are also easy to implement and can adapt to variable step sizes and orders. However, BDFs also have some drawbacks, such as potential instability for non-smooth or discontinuous problems, difficulty in starting the method, and sensitivity to round-off errors.

5 Runge-Kutta methods

Another popular class of methods for stiff ODEs is Runge-Kutta methods, which are single-step methods that use intermediate stages to compute the next value. Runge-Kutta methods can be explicit or implicit, and they can achieve high order of accuracy and efficiency. However, explicit Runge-Kutta methods are not A-stable, and implicit Runge-Kutta methods can be very complex and expensive, especially for high-order or high-dimensional problems. Some special types of Runge-Kutta methods that are designed for stiff ODEs are diagonally implicit Runge-Kutta (DIRK) methods, which have simpler implicit stages, and Rosenbrock methods, which use partial Jacobian information to reduce the nonlinear systems.

6 Stiffness detection and switching

Sometimes, you may not know in advance whether your ODE system is stiff or not, or how stiff it is. In such cases, you may want to use a numerical method that can detect and adapt to the stiffness of the problem, and switch between different methods or parameters accordingly. This can improve the accuracy, efficiency, and robustness of the numerical solution. Some criteria for stiffness detection are based on the local error estimate, the step size control, the convergence rate, or the eigenvalue analysis. Some strategies for switching are based on the error control, the stability region, the computational cost, or the user preference.

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?