Replace negative values with zero and values above 1 with 1, using loop in a 8760 x 1000 matrix

Dear all,

I have a simple question that I can not solve.

I have a 8760 x 1000 matrix in which some values are negative, and some are larger than 1. The matrix represents 1000 simulation outcomes for 8760 hourly wind energy capacity factors in a year. I want to replace the negative values with zero, and the values that are larger than 1 with 1. Also I want to count how many values I have replaced in any given run.

So far I have been first counting values that are above 1 and below 0 for the entire matrix using the numel function. This gives me just one number for the entire matrix. However I would like to obtain the number of values below 0 and above 1 for all of the 1000 simulations. Then after I get this I want to run a loop for replacing these values with 0 (when beloe zero) and 1 (when above one).

So far this is the code I have been using:

%DATA CORRECTION

%count data we correct
CountIF_exc_1 = numel(SimWind_CapFac_UP(SimWind_CapFac_UP>1));
CountIF_neg = numel(SimWind_CapFac_UP(SimWind_CapFac_UP<0));

%correct data
SimWind_CapFac_UP_corrected = zeros(SimHours,numberSim);
for y = 1:numberSim
    for x = 1:SimHours
        if SimWind_CapFac_UP_corrected(x,y) > 1
           SimWind_CapFac_UP_corrected(x,y) = 1;
        elseif SimWind_CapFac_UP_corrected(x,y) < 0
           SimWind_CapFac_UP_corrected(x,y) = 0; 
        end  
    end
end
  

This is however resulting in all zeros.

In summary could you please help me develop code to: a) count values that are below 0 and above 1 for each individual simulation run (the 1000 runs) and b) loop through the 8760 x 1000 matrix to replace these values with either 0 or 1.

Thanks a lot!

Answer 1

[Chris]

Is there a reason you want to do it in a loop?

Matlab excels at vectorized operations. The following is simpler and faster than a loop:

negmask = SimWind_CapFac_UP(:)<0;
excmask = SimWind_CapFac_UP(:)>1;
CountIF_neg = sum(negmask);
CountIF_exc_1 = sum(excmask);
NEWMATRIX = SimWind_CapFac_UP;
NEWMATRIX(negmask) = 0;
NEWMATRIX(excmask) = 1;

[Mak Dukan]

Hi!

I applied you code but I keep getting an error.

Undefined function or variable 'SimWind_CapFac_UP_corrected'.

Error in CorrectingCapFac (line 1)
negmask = SimWind_CapFac_UP_corrected(:)<0;

I need to first define the matrix I am creating, and then fill its values bt refferencing to the matrixSimWind_CapFac_UP_corrected

[Chris]

I apologize -- I hadn't fully understood your code. I attempted to correct mine, but it would be better if you understand what we're trying to show you.

[Chris]

A = randi([-1, 2], 4) % Some matrix

A = 4×4

1 -1 -1 0 -1 2 2 -1 0 0 1 0 0 1 0 2

count_neg = A(:)<0 % Matrix with all negative elements identified 

count_neg = 16×1 logical array
   0
   1
   0
   0
   1
   0
   0
   0
   1
   0

% The colon (:) returns all elements of the matrix in a column, ordered
% top-to-bottom, left-to-right.

% Add up all the elements of count_neg.
sum(count_neg)

ans = 4

% Change the negative elements
A(count_neg) = 9

A = 4×4

1 9 9 0 9 2 2 9 0 0 1 0 0 1 0 2

[Mak Dukan]

Hi!

This worked. I have however slightly changed the code so that it counts the values to be changed per row. Here is the code I ended up using:

negmask = SimWind_CapFac_UP <0;
excmask = SimWind_CapFac_UP >1;
CountIF_neg = sum(negmask_v2);
CountIF_exc_1 = sum(excmask_v2);
NEWMATRIX = SimWind_CapFac_UP;
NEWMATRIX(negmask) = 0;
NEWMATRIX(excmask) = 1;

I have a conceptual questions regarding the code

1) How come that this turns the values below 0 into 1 negmask = SimWind_CapFac_UP(:)<0;. I thought that you would need to use loops for that and did not know you can do it this way.

2) I also do not understand how it turns values in the NEWMATRIX to zero using this code NEWMATRIX(negmask) = 0. I thought that It can not do that, seeing that negmask is a 8760000 x 1 matrix and NEWMATRIX = SimWind_CapFac_UP is a 8760 x 1000 matrix

Thanks,

Mak

[Chris]

For future reference, to sum across rows, sum() allows a second argument indicating the direction in which to sum: sum(negmask,2)

1) For a typical language you would need to use loops, but as matlab was designed for matrix math, it has many features that make working with matrices and vectors easier, and usually faster than a loop. For a matrix A, A < 0

returns a matrix of the same size as A, with all elements less than zero identified. I added the colon (:) to return a vector, because the intent was to sum over all elements. Another method (which I probably should have used here instead) would be:

A = randi([-1, 2], 4);
count_neg = A<0

count_neg = 4×4 logical array
   0   0   0   0
   0   1   1   0
   0   0   0   1
   0   0   0   0

sum(count_neg,'all')

ans = 3

This sums all elements of the matrix, rather than all rows or all columns.

A(count_neg) = 9

A = 4×4

0 2 0 2 1 9 9 1 2 1 0 9 2 1 2 1

2) Arrays have a standard counting order--across each dimension, in order. Even with 3-dimensional or higher arrays, you could move through all elements one at a time without considering the dimensions. A 2D, mxn matrix is counted along columns (the first dimension), followed by rows.

vec = 1:9

vec = 1×9

1 2 3 4 5 6 7 8 9

reshape(vec,3,3)

ans = 3×3

1 4 7 2 5 8 3 6 9

As long as a vector has an equal number of elements as the array to which it is being compared, Matlab will iterate through the matrix in that order, comparing (or operating on) the nth element of vector x with the nth element of matrix A.

Does that help a little?

Also, for counting by columns, I think Stephen's answer does what you want in an even more condensed fashion.

Answer 2

[Stephen23]

Much simpler and more efficient than using loops, where A is your array:

ltz = sum(A<0,1);
gto = sum(A>1,1);
B = max(0,min(1,A))