How can I track time periods under certain criteria?

Hello everyone!

I would appreciate your help on this. I have a dataset of daily counts of temperature (please find attached) for 11 years, 2006-2016.

What I want to do is to find all time periods that demonstrate three or more consecutive days with temperature higher than the 95th percentile (this way I am trying to locate heat waves). I also need the respective dates.

What I have tried so far

m=1;
days=0;
for i = 1: length(Daily_Meteo.Daily_T)
if Daily_Meteo.Daily_T(i)> 28
days = days + 1;
if days >= 3;
HWs(m,:) = Daily_Data.Daily_DATE(i);
m=m+1;
end
end
end

The problem is that HWs(m,:) holds all days with T>28oC, but I want only 3 or more consecutive days...

Thank you in advance!

PS. I am on R2019b.

Answer 1

[Duncan Po]

You can put your data in a timetable and then use movsum like this:

tt = readtimetable('Daily_Data_Thessaloniki.xlsx');

tt.threehotdays = (movsum(tt.Daily_T > 28, [2 0]) == 3) ; % find all days when it and the previous 2 days are all above 28 degrees

[Daphne PARLIARI]

Hi Duncan!

Can you please explain what the line tt.threehotdays does?

[Duncan Po]

Assuming 28 degrees is the threshold, first compare the daily temperatures against this threshold:

tt.Daily_T > 28

This will return a logical array whose element is true for the days above 28 degrees and false otherwise.

Run a moving sum over this logical array, with a window length of 3 including the current element and the previous 2. Each windows inspects the current day and the two previous days, which is the 3 consecutive days:

movsum(tt.Daily_T > 28, [2 0])

Since the input is logical, the values are 0s and 1s. If 3 consecutive days are above 28 degrees, all 3 values will be 1, so the moving sum on that day will be 3. We check for when moving sum is 3:

movsum(tt.Daily_T > 28, [2 0]) == 3

[Daphne PARLIARI]

I got it, thanks!

Nevertheless, on a 3-day long Heatwave, only the last day gets index 1. The first two days are always 0 (e.g. 27-29/6/2006)...

What is the problem with that?

[Duncan Po]

movsum in this case only looks at the current day and the previous two. If you want to the first two days in a 3-day long heatwave to also return true, you need to look at the next two days as well. For example, the first day of a heat wave should be identified by checking the current day and the next two days are all above the threshold.

You can either use a FOR loop (more efficient but more code to write) or use multiple movsum:

previoustwo = (movsum(tt.Daily_T > 28, [2 0]) == 3);
nextandprevious =  (movsum(tt.Daily_T > 28, [1 1]) == 3);
nexttwo = (movsum(tt.Daily_T > 28, [0 2]) == 3);
tt.threehotdays =  previoustwo | nextandprevious | nexttwo;  % elementwise or

[Daphne PARLIARI]

Thank you Duncan.

Is there a way to also count the number of HW episodes?

[Duncan Po]

Use diff:

d = diff(tt.threehotdays) == 1;  % d is 1 at transition from 0 to 1
loc = find(d) + 1;   % +1 because diff output starts at 2nd position