What is the best way to turn a time vector into something plottable when you have large quantities of data?

What I need: I need to sequentially plot a calculated value vs. time. and have the x-axis be recognized as a date.

The question: What is the best way to turn a time vector into something plottable?

The problem: I have over 5 million data points. This current code took 1.5 hours to get through %15 of the data. And I will need use this code with additional (possibly more) data at a later time.

What I have: The date in 6 columns [Year, Month, Day, Hour, Minute, Second] The array type is a double. I have over 5 million data points.

Just datenum() allows me to plot it correctly but the axis is unrecognizable as a date.

My current code:

for i= 1:length(Data)  
    % Other operations
    if p1~=0 && p2~=0 % Unrelated to date. 
        % Other operations

        date= datenum([Data(i,1),Data(i,2),Data(i,3),Data(i,4),Data(i,5),Data(i,6)]);
        t(i,1) = datetime(date,'ConvertFrom','datenum'); % in 1.5 Hours got through %15 of the data...
    end
end

Is there a more efficient way to turn the data into a datetime for plotting? OR is there a way that datenum() can be plotted to show the dates?

Thank you for any and all advice!

[dpb]

Yes, to both, but use datetime instead of the archaic and deprecated datenum

plot and friends are datetime aware so you get the time axes for free.

However, plotting 5M points is silly; at most there are only about 2000 pixels on a 24" monitor so you can effectively resolve only about 0.04% of the data points; all the rest will be plotted on top of each other assuming a reasonably regular time sampling.

So, start by decimating your data by at least a factor of 1000 (2000 or 2500 would more closely match available resolution) for plotting purposes; you could minmax it to ensure the plot does cover the range of the data for visual representation.

[dpb]

That is,

ix=((p1~=0) & (p2~=0));   % NB: if are logical already, then can do away w/ explicit condition
t=datetime(Date(ix));
% Other operations
...

[Jayne]

I hadn't considered that I could (and probably should) reduce the amount of data for plotting. And the logical indexing is also a great idea.

I've gotten the time down. I'm now going from the time vector matrix directly into dattime outside of the for loop. Which has cut the time down from 10 hours to 30 seconds (and thats without removing any data)!

Thanks for the help!

Answer 1

[Walter Roberson]

filter the entire data by p1 and p2 giving you 6 columns. pass the 6 to datetime, and it will do a mass conversion. You do not need to use datenum.

[Jayne]

Thank you!

My code currently looks like this:

% Create time vector matrix in seperate for loop (with other operations)
for i= 1:length(Data)
    date(i,:)= [Data(i,1),Data(i,2),Data(i,3),Data(i,4),Data(i,5),Data(i,6)];
end
ix=((p1~=0) & (p2~=0));
t= datetime(date(ix));

plot(t,C(:,1))

[dpb]

There's no need for the loop or temporary variable --

t= datetime(Data(ix,1:6));

MATLAB is MATrix LABoratory, after all -- vector addressing works in all dimensions, not just the first. :)

Of course, if you have reason for the full datetime array for other purposes, then

tdt= datetime(Data(:,1:6));

and then select the subset along with whatever other data need for analysis with the logical addressing vector.