how to merge interpolated data into an existing timetable?

Hi,

i have a long uni variate timeseries data with 1 minute resolution. Here is an attachment of one part of this data set,shifted into timetable.

this attached data has one time slot ( 28 Mar 2017 21:49:00 to 10 Apr 2017 10:28:00) having negative value, which should not be negative. so i have converted this negative value to NaN, and then this timerange data is interpolated. i have succesfully interpolated this timeslot data. but i am not getting a way to put this interpolated data into the existing time table. caz i need full timetable at the end, with the interpolated values as well . Here is my code, please can any one guide me by looking at my code, what am i missing?

one thing more, when i run this code , it displayed clearly the interpolated data with the existing data in the plot, but with a warning "Warning: Columns of data containing NaN values have been ignored during interpolation. " how to get rid of this warning?

Looking forward to any guidance anxiously...

load OneMinute_ObsData;
plot(OneMinute_ObsData.timmendorf_time,OneMinute_ObsData.timmendorf_waterlevel);

for i= 1:size(OneMinute_ObsData,1)
                if( OneMinute_ObsData.timmendorf_waterlevel(i,1)< 0)        
                          
                   OneMinute_ObsData.timmendorf_waterlevel(i,1) = NaN;
                end
end

Timerange_for_interpolation=[datetime('2017-3-28 21:49:00'):minutes(1):datetime('2017-4-10 10:28:00')]';

V = interp1(OneMinute_ObsData.timmendorf_time, OneMinute_ObsData.timmendorf_waterlevel, Timerange_for_interpolation, 'spline');
% Plot the interpolated points.

hold on
plot(Timerange_for_interpolation,V,'r');

Answer 1

[Peter Perkins]

Really this is best done with fillmissing. It's just two lines:

>> X = [1;2;-3;-4;5;6];
>> tt = timetable(X,'RowTimes',datetime(2018,12,27,0,0:5,0))
tt =
  6×1 timetable
            Time            X 
    ____________________    __
    27-Dec-2018 00:00:00     1
    27-Dec-2018 00:01:00     2
    27-Dec-2018 00:02:00    -3
    27-Dec-2018 00:03:00    -4
    27-Dec-2018 00:04:00     5
    27-Dec-2018 00:05:00     6
>> tt.X(tt.X<0) = NaN
tt =
  6×1 timetable
            Time             X 
    ____________________    ___
    27-Dec-2018 00:00:00      1
    27-Dec-2018 00:01:00      2
    27-Dec-2018 00:02:00    NaN
    27-Dec-2018 00:03:00    NaN
    27-Dec-2018 00:04:00      5
    27-Dec-2018 00:05:00      6
>> tt1 = fillmissing(tt,"spline")
tt1 =
  6×1 timetable
            Time            X
    ____________________    _
    27-Dec-2018 00:00:00    1
    27-Dec-2018 00:01:00    2
    27-Dec-2018 00:02:00    3
    27-Dec-2018 00:03:00    4
    27-Dec-2018 00:04:00    5
    27-Dec-2018 00:05:00    6

[bushra raza]

Thank you so much sir.

since many days, i am working on it, and even then i interpolated the values, but my data set was showing some missing data in summary(timetable) command. and i was not getting a way out.

your solution is superb, really worked for me. :)

Sir, can you please guide me in my next step?

i have to calculate the trend of my timeseries data.

this one minute resolution data is further meged in an hourly data, and the whole data set is now in hourly resolution.

please give me some hint, i tried box plot and linear trend line , but unfortunately i could not get a way to interpret it.

how to calculate the trend values for my timeseries data set?

[Peter Perkins]

That's a different question that deserves its own thread.

Answer 2

[bushra raza]

Hi,

i have got the solution myself. just few more lines of code.

here is the code, May it be helpful for some one else as well.

load OneMinute_ObsData;
% plot(OneMinute_ObsData.timmendorf_time,OneMinute_ObsData.timmendorf_waterlevel);

for i= 1:size(OneMinute_ObsData,1)
                if( OneMinute_ObsData.timmendorf_waterlevel(i,1)< 0)        
                          
                   OneMinute_ObsData.timmendorf_waterlevel(i,1) = NaN;
                end
end

timmendorf_time=[datetime('2017-3-28 21:49:00'):minutes(1):datetime('2017-4-10 10:28:00')]';
timmendorf_waterlevel = interp1(OneMinute_ObsData.timmendorf_time, OneMinute_ObsData.timmendorf_waterlevel, timmendorf_time, 'spline');
%make a new timetable of interpolated values
TT = timetable(timmendorf_time,timmendorf_waterlevel);
OneMinute_ObsData = [OneMinute_ObsData ;TT];
%%Sort in Time Order
sorted = issorted(OneMinute_ObsData); % logical 0
    if (sorted == 0) 
        OneMinute_ObsData = sortrows(OneMinute_ObsData);
    end %if end   

plot(OneMinute_ObsData.timmendorf_time,OneMinute_ObsData.timmendorf_waterlevel);