Condition number of empty matrix

if is a 0 x 0 matrix, then cond( A ) returns 0. However, we know that the condition number is always >= 1. Also, one could argue that the empty matrix is an identity matrix, and hence its condition number should equal 1.

Any particular reason why matlab made this choice?

[jessupj]

(unrelated, but i'm curious how you would argue that a null object is the identity matrix. what space is it the identiy on?)

[jessupj]

possibly related: i improved my understanding of how matlab performs ops on empty objects from this disucssion: https://www.mathworks.com/matlabcentral/answers/1647265-sum-on-empty-arrays

[Robert van de Geijn]

Exactly the question I would expect! (Not that I have a prepared answer)

A couple of ways in which I would argue this:

1) I created the matrix as A = eye( 0,0 ).

2) A x = x is true for any vector (since the vector x is 0 x 1 ).

I would think that cond( A ) should be undefined if A is 0 x 0.

[jessupj]

I agree that cond([]) should be undefined. It needs a ratio of eigenvalues, and it's not clear to me ew's are well-defined for operators on R^0 (or whatever it is here).

You're working in the space of zero-tuples, whose only element (pardon my non-algebraic mindedness) is a unique empty set. it's not actually an empty n-typle as in the argument above ( A is 0x0, x is 0x1). It seems that formally you'd have operator A=[] (just empty, not actually 'empty x empty') and x = [], right?

Distinguishing them using empty vs. singleton dimensions feels completely 'artificial' or at least 'numerical' (but I do like the argument for why MATLAB shouldn't give you a zero).

[David Goodmanson]

Hi Robert

I don't believe 0 is a satisfactory result, but since the condition number is the ratio of the largest to smallest singular value, you can make a good case that the result should be [ ].

A = [];
s = svd(A)
     s =  0×1 empty double column vector
isempty(s)
     ans =   logical
             1
condvalue = max(s)/min(s)
     condvalue =   []

% the simpler variable [] gives the same ratio

% size([])
     ans = 0     0
[]/[]
     ans =   []

s has some funky dimensions, but it's still empty, and the resulting condition number equals [ ].

I think you could also make a good case for the condition number equaling NaN, since the ratio of the size of two empty sets seems ill defined.

[Robert van de Geijn]

I don't think you should think about it that way.

The condition number is the maximum by which a vector is stretched divided by the minimum that a vector is stretched. The only vector that exists is the 0 x 1 vector. When you apply a 0 x 0 matrix to it, you get that same vector back, regardless. Hence the condition number, if defined at all, should equal 1.

[Robert van de Geijn]

Let me now complicate matters even more with some more musings...

However, if anything, x being a 0 x 1 vector (a vector with no elements), its length (2-norm) must be 0, and hence it is the zero vector. Thus, the 2-norm of a 0 x 0 matrix cannot be defined...

If the 2-norm cannot be defined, then the condition number cannot be defined...

Answer 1

[Christine Tobler]

The case of a 0-by-0 matrix doesn't have any very useful definition, as you note correctly in the comments above.

MATLAB does what it does because it computes any p-condition number using the formula:

norm(A, p) * norm(inv(A), p)

and of course the norm of [] is 0, as is the norm of the inverse of [].

A legitimate question could be if the norm of a [] matrix should be 0, or if it should be NaN since this matrix can't be mulitplied with a vector that has norm 1. But in practical terms, I think it's more useful to define this norm as being 0 than returning NaN.

[John D'Errico]

Just thinking, I'll play the devil's advocate here, arguing it should return NaN.

What does it mean, when a result comes back as NaN? NaN means to me that a computation could have taken on any of multiple values, and probably infinitely many values. For example, what is sin(inf)? If we try to take the limit of sin(X), as x approaxhes infinity, clearly it could be any value from the closed interval [-1,1]. And so we assign a NaN to that result.

Similarly, what is inf/inf? Again, we can answer that only in context of a limit. But what matters is how fast you approach that limit in the numerator and the denominator. For example, we could ask for the limit of (x/x^2), as x approaches infinity. Clearly that would be 0. But suppose we tried to take the limit of 17*x/x, as x approaches inf? Now we could argue that inf/inf should be the number 17. The point is, inf/inf can only be returned as the not-a-number result of NaN, because we could write it as any limits we choose, and we can get infinitely many different results. 0/0 is the same thing, even though there are passionate pleas to be found online for 0/0 to have some specific value, usually 1.

The point being, a NaN should be returned when a result could arguably be anything.

Is that a valid argument to apply to cond([])?

Christine makes a valid argument that it should be an empty result, since norm([]) is empty and inv([]) also is empty. Each of these follow the MATLAB convention to return an empty output for empty input. And of course,

[]/[]

ans =

     []

So perhaps what we need to argue is if norm([]) should be zero? Should inv([]) be something other than []?

In both of those cases, it makes sense they should be []. Consider inv. inv(A) is a square matrix of the same size as A. So inv([]) CANNOT be a 1x1 scalar. It MUST be []. Similarly, what is norm([])? The 1-norm would have us take the sum of the absolute values of the sum of the elements of the input. That looks like [] to me, and I could make the same argument for norm([]). (Feel free to pick your favorite norm. Mine is Norm Abrams.)

Your honor, the devil's advocate would like to approach the bench, and ask to be removed from the case.

[Christine Tobler]

I think both cond and norm should always return a scalar, based on the definition (a norm projects to the space of nonnegative numbers, and a condition number is just the product of two norms). Also purely programmatically, a function that nearly always returns scalar but rarely returns empty would be quite painful and best avoided.

With norm, things are a bit weirder in the empty case:

• norm(zeros(0, 1)) is the euclidean norm of a length-0 vector, so clearly this should return 0.
• norm(zeros(0, 0)) however is the operator norm, defined as max_x norm([]*x) / norm(x), where x is a 0-by-1 vector, and you can make a very good argument that this would be 0/0 = NaN.

However, I think norm would not practically be more useful if we had the behavior NORM of a 0-by-1 empty returns 0 and NORM of a 0-by-0 empty returns NaN, even though it would stick more closely to the definitions. That seems like a pitfall that many could fall into who like to just use an [] when they really mean a vector of length zero.

[Robert van de Geijn]

This is all an entirely academic discussion. As the person who originally posed the question, I am perfectly happy to leave it as the status quo. No need to push this any further.

[Paul]

@Christine Tobler

norm(zeros(0, 1)) is the euclidean norm of a length-0 vector, so clearly this should return 0.

Is "length" being used here in the mathematical sense, i.e., , or in the Matlab sense, i.e., length(x)?

If the former, I have a concern in that I'm not aware of a mathematical definiton of an empty vector and so don't think it clearly has a mathematical norm or Matlab norm() of 0. I always thought that the empty matrix was basically a TMW construct.

Side note. I'm paraphrasing here, but very, very early Matlab documentation had a statement like: We don't know if we've implemented the empty matrix correctly, but we've found it to be a very useful concept.

Of course, Matlab's implementation of the empty matrix concept has evolved quite a bit since then.

[Christine Tobler]

I meant the MATLAB sense, so length(x) == 0 - the extension of a sum over all elements is to return 0 for the sum over zero elements, and this translates to the euclidean norm.

Here's a blog post about computing with empty arrays:

https://blogs.mathworks.com/loren/2009/11/04/calculus-with-empty-arrays/

There are definitely some hard-to-define cases coming up there, and I don't think the concept is mathematically defined, yes. However, it is very useful in practice.

[Steven Lord]

I always thought that the empty matrix was basically a TMW construct.

No, it isn't. See the book linked in the last paragraph of this blog post from Professor Nick Higham and the excerpt from it quoted in that post.

[Christine Tobler]

Very nice link, Steve! The paper cited in that post ("An empty exercise", de Boor, 1990), has the following to say on norm([]):

"Further, any norm of an empty matrix is zero, as the supremum of the empty set of nonnegative numbers. This implies that the condition number of the square empty matrix [] is 0. This makes [] the only matrix with a condition number less than 1 and the only invertible matrix with zero norm."

[Paul]

@Steven Lord, thanks for the link.

My assumption that TMW invented Matlab's [original] empty matrix implementation was based on the actual statement in the doc at that time (from that link's link to deBoor):

"'As far as we know, the literature on the algebra of empty matrices is itself empty. We're not sure we've done it correctly, or even consistently, but we have found the idea useful."

Perhaps I should have said: I always thought that the TMW implementation of Matlab's empty matrix was basically a TMW construct.

No matter, obviously. I just have an historical interest.

[Robert van de Geijn]

As a purist, I consider a matrix to strictly be a convenient representation of a linear transformation. So, the first questions are

1) is R^0 (or C^0) a vector space?

2) Are there linear transformations that map R^0 to R^0?

3) How are these linear transformations represented as (0 x 0) matrices?

That establishes whether [] is even a matrix.

Let's say that the answers are

1) Yes, and its only element is the 0 x 1 vector, which then must be the zero vector, since the zero vector must be in the space.

2) Yes, only one: the linear transformation that maps the 0 x 1 to itself.

3) The matrix that represents it then is [] (the 0 x 0 matrix).

At this point we can start discussing the norm (let's stick to the 2-norm).

a) Since the 0 x 1 vector is the zero vector, its norm equals 0.

b) While the 2-norm for n > 0 is defined as max_x \neq 0 || A x ||, it is better to go back to thinking about linear transformations. The "size" (2-norm) of a linear transformation really measures by how much the linear transformation stretches the vector to which it is applied. One can argue that the linear transformation

a) maps the 0 x 1 vector to the zero vector and hence || A || = 0

b) maps the 0 x 1 vector to itself and hence A = I and || A || = 1

c) stetches the 0 x 1 vector by any nonnegative integer, since it is the zero vector.

The fact that all of these can be reasonably argued to me indicates that the condition number is ill-defined. Hence, I believe NaN or "not defined" are the only results in the running.

Now, it may be that since matlab is used as a tool for manipulating arrays in addition to matrices, it is convenient to define it differently. But then we should not really call it the condition number. It is merely the matlab operation that is denoted by cond().