invers from covariance of a matrix*matrix'

given a is a matrix, is the matrix of covariance of (a*a') is always singular?

[the cyclist]

Can you please clarify? Are you interested in the singularity of cov(a) for arbitrary a, or of cov(b), for b = (a*a')?

[eri]

cov(b) for b=a*a'

Answer 1

[Teja Muppirala]

cov(a) is ALWAYS singular for ANY square matrix a, because you subtract off the column means. This guarantees that you reduces the rank by one (unless it is already singular) before multiplying the matrix with its transpose.

a = rand(5,5); % a is an arbitrary square matrix
rank(a) %<-- is 5

a2 = bsxfun(@minus, a, mean(a));
rank(a2) %<-- is now 4

cova = a2'*a2/4   %<-- (rank 4) x (rank 4) = rank 4
cov(a)  %<-- This is the same as "cova"
rank(cova) %<-- verify this is rank 4

Answer 2

[the cyclist]

a = [1 0; 0 1]

is an example of a matrix for which (a*a') is not singular.

Did you mean non-singular?

[Titus Edelhofer]

But cov(a) is indeed singular ... The examples I tried so far have a singular covariance but I haven't found a way to prove it's always the case ...

[Walter Roberson]

We uses inverse covariance a lot in our work, and we do not find them to be singular provided that the number of rows is at least the (number of columns plus 1).

[Titus Edelhofer]

Interesting. I was wondering already what sense inverse covariances might have. I tried with quadratic a's only.

[eri]

what i asked is cov(a*a')

cov (a) may not always singular and a*a' may also not always singular, but cov(a*a')?

[Walter Roberson]

a*a' is always square, and if my recollection is correct, cov() of a square matrix is singular.

[Titus Edelhofer]

@Eri: trusting Walters recollection is usually a good bet ;-)

[Walter Roberson]

Just don't ask me _why_ it is singular. I didn't figure out Why, I just made sure square matrices could not get to those routines.

[Walter Roberson]

Experimentally, if you have a matrix A which is M by N, then rank(cov(A)) is min(M-1,N), and thus would be singular for a square matrix.