Element wise mean produces an incorrect result

Hi,

I have a x*n*t array. The array represent some atmospheric measure at location (x,y) and time t. 'field_example' (attached) is an example of field at t=1; I want to compute the time averaged field. So easy, just do

if true
  mean_array = mean(array,3)
end

My problem is that doing so produces the 'problem_field' plot. As you can see this can't be the mean of an atmospheric field as there's no spatial correlation between values. I found out that what was giving that problem was the last time field, that is when t=T. But my field at time T is actually pretty normal. Proof of it, and the strangest thing, is that if do

if true
  tmp = mean(array(:,:,1:end-1),3);
  mean_array = (tmp+array(:,:,end))./2;

end

It produces the 'correct_field'!

Any idea of why it is doing that?

[David Goodmanson]

Hi manuel, not a surfeit of information here, but one thing going on is that the mean( ... ,3) of the entire array weighs every layer equally with weight 1/n, but your second calculation gives the last layer a weight of 1/2 and each of the others a weight of 1/(2(n-1)). So it's a different calculation, with the last layer getting the lion's share of the credit. Perhaps it's a different layer that is acting up.

[manuel FOSSA]

Hi David, Yes, the second computation is different from the first, i just wanted to say that the last layer is not a faulty one. I'll investigate according to your remark but if i do the mean over all layers except the last, a correct mean layer is produced. Thanks for the remark though.

[John D'Errico]

In fact, it is not at all obvious that what you did was correct. What you did may LOOK correct, but only looks right because it is smoother.

Be very careful in assuming that something is correct, just because it looks like what you wanted to see. If mean does not do what you wanted, then are you need to spend some serious effort in looking into what is happening here.

It would be far easier to provide help if you show more information. Providing the actual array as an attached .mat file would surely help a bit.

[Bjorn Gustavsson]

As David suggested some other layer might "act up". You might quickly investigate the temporal behaviour thisly:

subplot(2,1,1)
imagesc(squeeze(array(:,12,:))),colorbar
subplot(2,1,2)
imagesc(squeeze(array(12,:,:))),colorbar

Then you might catch the offending layer, unless the number of time-slices are very large...

HTH

[manuel FOSSA]

Dear John, Bjorn

I attached the array as well as a subplot of all slices. I can't spot any slice that would act up just by looking at each field. They all have coherent spatial structure.

[David Goodmanson]

hello manuel, What's going on is that for whatever reason, the sum in the third dimension of all your slices equals zero within numerical error. The 71x46 different sums are all down around 1e-15 and the elements themselves are of both signs, as if someone has subtracted off the mean already. If you sum all except for one of these you get a reasonable result, but if you sum them all you get a plot of autoscaled numerical confetti.

[manuel FOSSA]

Thank you David, i need to check why the sum is zero..this is very surprising.

David Goodmanson is accepted answer. (please feel free to put it as answer)

[David Goodmanson]

thank you manuel for your answer offer, I did that.

Answer 1

[David Goodmanson]

hello manuel, What's going on is that for whatever reason, the sum in the third dimension of all your slices equals zero within numerical error. The 71x46 different sums are all down around 1e-15 and the elements themselves are of both signs, as if someone has subtracted off the mean already. If you sum all except for one of these you get a reasonable result, but if you sum them all you get a plot of autoscaled numerical confetti.