Iterative methods for linear systems were invented for the same reasons as they are used today,namely to reduce computational cost. Gauss states in a letter to his friend Gerling in 1823: 'you will in the future hardly eliminate directly, at least not when you have more than two unknowns'.
Richardson's paper from 1910 was then very influential, and is a model of a modern numerical analysis paper: modeling, discretization, approximate solution of the discrete problem,and a real application. Richardson's method is much more sophisticated that how it is usually presented today, and his dream became reality in the PhD thesis of Gene Golub.
The work of Stiefel, Hestenes and Lanczos in the early 1950 sparked the success story of Krylov methods, and these methods can also be understood in the context of extrapolation, pioneered by Brezinski and Sidi, based on seminal work by Wynn.
This brings us to the modern iterative methods for solving partial differential equations,which come in two main classes: domain decomposition methods and multigrid methods. Domain decomposition methods go back to the alternating Schwarz method invented by Herman Amandus Schwarz in 1869 to close a gap in the proof of Riemann's famous Mapping Theorem. Multigrid goes back to the seminal work by Fedorenko in 1961, with main contributions by Brandt and Hackbusch in the Seventies.
I will show in my presentation how these methods function on the same model problem ofthe temperature distribution in a simple room. All these methods are today used as preconditioners for Krylov methods, which leads to the most powerful iterative solvers currently knownfor linear systems.[-]