Is there a way of getting this done more efficiently ?

I have the following code and it is taking a very long time to go through it... I dont think its an infinite loop but it is as follows:

    Y = zeros(1069,30658);
    D1 = LagOp({0,1,1,1},'Lags',[0,1,2,1]);
    for n = 2:30658;
        for j = 2:1063
            if filter(D1,Ret((D1.Degree + j),n),'Initial',Ret(2:D1.Degree,n)) < 0;
                Y(j+1,n) = -1*Ret(j+1,n);
            else
                Y(j+1,n)=Ret(j+1,n) ;
            end
        end
    end

Basically I want to flip the sign of the current element in the matrix if the previous 3 elements before it add up to being less than 0. Otherwise to leave it alone. Could it be the ... else statement causing the trouble here ?

Thanks,

[Walter Roberson]

To check: once you have flipped the sign on a value, it is that new value that will be included in the calculations for the next 3 elements?

I ask because the alternative is much faster, where it is the old value that is used for the sum -- that could be done relatively easily. Your current code does not use the updated element.

[Putsandcalls]

Do you mean as the following ? for n = 2:30658;

Y = zeros(1069,30658);
D1 = LagOp({ *1*,1,1,1},'Lags',[0,1,2,1]);
  for j = 2:1063
          if filter(D1,Ret((D1.Degree + j),n),'Initial',Ret(2:D1.Degree,n)) > 0;
              Y(j+1,n)=Ret(j+1,n) ;
          else
              Y(j+1,n) = -1*Ret(j+1,n);
          end
      end
  end

I also changed the element for if conditions since I would probably have more positive than negative. Is there a way of not even specifying a filter to complete the job ?

Thanks

[Walter Roberson]

Nothing to do with your lag structure. Going by your "basically" description,

Y(:,n) = Ret(:,n);
t = conv(Y(:,n), [1 1 1], 'valid');
mask = [false(3,1); t(1:end-1)<0];
Y(mask,n) = -Y(mask,n);

Half of this code is edge case for not processing the first 3 items because there would not be 3 prior values to sum for them.

The for loop equivalent to the above is

   Y(:,n) = Ret(:,n);
   for j = 4 : size(Y,1)
     if sum(Ret(j-3:j-1)) < 0   %looking at R
       Y(j,n) = -Y(j,n);
     end
   end

This corresponds to the fact that in your existing code, you are always looking at Ret (unchanged) as you move on, so any change you make to Y does not affect your later choice.

Now compare:

   Y(:,n) = Ret(:,n);
   for j = 4 : size(Y,1)
     if sum(Y(j-3:j-1)) < 0    %looking at Y
       Y(j,n) = -Y(j,n);
     end
   end

In this slightly different version, if you change the sign of (say) Y(8) then when you move on to Y(9) it would be the changed Y(8) that you would be adding to Y(6) and Y(7)

For example, consider -1 -2 -3 -7 -5 . Examine the position with the -7 . The sum of the -1 -2 -3 is negative so you change the sign of the -7 leading to -1 -2 -3 +7 -5 . Now when you move on to look at the -5 position, do you add the values that were originally there, -2 -3 -7, and get a negative value so you flip -5 to +5? Or do you look at the changed value, -2 -3 +7, get a positive sum, and so leave the -5 as -5 ?

The vectorized version with conv works on the basis of what the original values were. Is that what you would want?

I have not tried to work out what those lags of yours are doing.

[Putsandcalls]

Yes sorry for being unclear on that part, it should be the fact that if we had 5 values -1 -2 -3 -7 -5, it follows that it would be the case where -1 -2 -3 +7, but for the next term, it would count -7 and so the result was +5. As a side note, when working with lagged structures of time series for example, wouldn't this mostly substitute the need for a filter ? I had read up the idea of filter on the documents page but I find that it is quite difficult to grasp the concept.

[Walter Roberson]

Some kinds of filters are implemented using convolutions.

[Putsandcalls]

I dont know why I keep getting an error saying that it has a dimension mismatch. So is the first part itself suppose to be a separate statement or is the entire 2 parts suppose to be a nested for loop? When I try the nested for loop, it tells me a dimension mismatch. While do a single for loop just gives me no changes.

Answer 1

[Walter Roberson]

Without taking lags into account,

Y(:,n) = Ret(:,n);
t = conv(Y(:,n), [1 1 1], 'valid');
mask = [false(3,1); t(1:end-1)<0];
Y(mask,n) = -Y(mask,n);

This version uses the original values in each step of the calculation, not the potentially-flipped values.

To do everything at once,

t = conv(Ret, [1 1 1].', 'valid');
mask = [false(3, size(Ret,2)); t(1:end-1,:) < 0];
Y = Ret;
Y(mask) = -Y(mask);

[Putsandcalls]

It works awesome, thanks ! dont know what the 2nd part does though :)

[Walter Roberson]

Which second part?

[Putsandcalls]

I mean about

if true
  t = conv(Ret, [1 1 1].', 'valid');
  mask = [false(3, size(Ret,2)); t(1:end-1,:)];
  Y = Ret;
  Y(mask) = -Y(mask);
end

But another question I had is how would you modify this to do it with changing the next 3 values instead of changing the value right after the sum is taken. So for example: -3 -2 -4 -5 6 8 9 1, then at the point -5, take the sum: -3 - 2 - 4 < 0 and then it would be 3 numbers ahead that changes signs. Thank you

[Walter Roberson]

Needs conv2() insted of conv. I keep forgetting how limited conv() is.

Also I had a typo,

t = conv2(Ret, [1 1 1].', 'valid');
mask = [false(3, size(Ret,2)); t(1:end-1,:) < 0];
Y = Ret;
Y(mask) = -Y(mask);

[Walter Roberson]

Sorry I am falling asleep now.

[Putsandcalls]

Thanks a lot, but if you have time to spare, I hope you can answer my question :)