Conjugate Gradient Method

Conjugate Gradient Method to solve a system of linear equations

The conjugate gradient method aims to solve a system of linear equations, Ax=b, where A is symmetric, without calculation of the inverse of A. It only requires a very small amount of membory, hence is particularly suitable for large scale systems.

It is faster than other approach such as Gaussian elimination if A is well-conditioned. For example,

n=1000;
[U,S,V]=svd(randn(n));
s=diag(S);
A=U*diag(s+max(s))*U'; % to make A symmetric, well-contioned
b=randn(1000,1);
tic,x=conjgrad(A,b);toc
tic,x1=A\b;toc
norm(x-x1)
norm(x-A*b)

Conjugate gradient is about two to three times faster than A\b, which uses the Gaissian elimination.