How can iterative solvers improve computational cost for numerical methods?

1 What are iterative solvers?

Iterative solvers are algorithms that use repeated steps to approximate the solution of a problem, rather than finding the exact solution in one step. For example, if you want to solve a system of linear equations Ax = b, where A is a large and sparse matrix, you can use an iterative solver to generate a sequence of vectors x0, x1, x2, ..., xn that converge to the true solution x as n increases. Iterative solvers are often preferred over direct solvers, which use matrix factorization or elimination techniques, because they require less memory and time for large and sparse problems.

2 How do iterative solvers work?

Iterative solvers work by using an initial guess x0 and applying a formula or a procedure to generate a better guess x1, and then repeating the process until a desired level of accuracy or convergence is reached. The formula or the procedure depends on the type of iterative solver, which can be classified into two categories: stationary and non-stationary. Stationary iterative solvers use a fixed formula that depends only on the matrix A and the right-hand side b, such as the Jacobi, Gauss-Seidel, or Successive Over-Relaxation methods. Non-stationary iterative solvers use a variable formula that changes at each iteration, such as the Conjugate Gradient, GMRES, or BiCGSTAB methods.

4 What are the challenges of iterative solvers?

Iterative solvers require careful analysis and selection due to their challenges and drawbacks. These solvers do not guarantee the existence or the uniqueness of the solution, nor do they have a fixed number of iterations or a predictable running time. Furthermore, iterative solvers may introduce numerical errors or instabilities, and they may require additional information or parameters. For example, the iterative solver may need a good initial guess, a suitable convergence criterion, a compatible preconditioner, or an optimal acceleration technique. These properties can affect the accuracy or stability of the solution and may not be easy to find or tune.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?