How to I interpolate only if 5 or less missing data next to each other?

Hi,

I have an array of data, I want to interpolate NaNs over columns but only if there is 5 or less consecutive NaNs in a column.

I was thinking of using isnan function, and then marking places where I need to interpolate, but I'm not sure exactly how to do that.

Any ideas are welcom :) Thanks!

K.

Answer 1

[Jan]

With http://www.mathworks.com/matlabcentral/fileexchange/41813-runlength :

[b, n]      = RunLength(isnan(x));
shortNaN    = RunLength(b & (n < 5), n);
x(shortNaN) = interp1(find(~shortNaN), x(~shortNaN), find(shortNaN), 'linear');

INTERP1 shows a warning then:

 Warning: NaN found in Y, interpolation at undefined values
          will result in undefined values.

But it is not possible that the NaN's appear in the output of the interpolation in this case and therefore the warning is misleading. To avoid it, you can either use a much faster replacement of INTERP1, see e.g. http://www.mathworks.com/matlabcentral/answers/64118#answer_75899 . Or you can interpolate on all NaNs at first and overwrite the long runs afterwards:

index    = isnan(x);
x(index) = interp1(find(~index), x(~index), find(index), 'linear');

[b, n]     = RunLength(index);
longRun    = RunLength(b & (n > 4), n);
x(longRun) = NaN;

[Karolina]

Thanks, that seems like a nice solution, but I have some problems with getting RunLength to work..

When I try to compile it (manually) I get this message:

Warning: You are using gcc version "4.7.3". The version currently supported with MEX is "4.4.x". For a list of currently supported compilers see: http://www.mathworks.com/support/compilers/current_release/

-> gcc -c -I/usr/local/MATLAB/R2013a/extern/include -I/usr/local/MATLAB/R2013a/simulink/include -DMATLAB_MEX_FILE -ansi -D_GNU_SOURCE -fexceptions -fPIC -fno-omit-frame-pointer -pthread -std=c99 -DMATLABVER=801 -O -DNDEBUG "RunLength.c"

RunLength.c:108:25: fatal error: RunLength.inc: No such file or directory compilation terminated.

    mex: compile of ' "RunLength.c"' failed.

Error using mex (line 206) Unable to complete successfully."

Any ideas how to solve this?

[Jan]

@Carolina: You are right, there is a missing file in the submission. To my surprise nobody complaint before since May 2013. What a pity.

I'm going to update the submission and in between you can use RunLength_M instead.

[Dolly More]

Does it work for a matrix array? I tried on a 389 x 1800 matrix. I am getting the following error.

Error using RunLength_M (line 81)

*** RunLength[m]: 1st input must be a row or column vector.

Also, does it only interpolate row-wise or column-wise as well?

[Jan]

The documentation clarifies exactly, that RunLength operates on vectors only:

help RunLength

The error message tells you the same. If you provide a vector as input, it is processed in the direction, it has: row-vectors are processed row-wise, column-vectors are processed column-wise, as usual in Matlab.

If you provide a matrix as output, each column (or row) can produce a different number of outputs. Therefore a cell array must be replied. Simply call RunLength in a loop and provide the rows or columns of your matrix.

Answer 2

[Chad Greene]

The interp1gap function allows you to specifiy a maximum gap length over which to interpolate.

Answer 3

[Image Analyst]

And if the run of nans is longer than 5, just leave them as nans? Do you have the Image Processing Toolbox, which makes it easy to find runs of certain lengths?

[Karolina]

Yes, if it's more than 5 leave the NaNs. Otherwise the interpolation doesn't make sense and my data is too "smooth". I have the image processing toolbox also.

[Image Analyst]

You can start with something like this:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

% Make signal
t = linspace(0,360, 50);
signal = sind(t);
subplot(2,2,1);
plot(t, signal, 'bo-', 'LineWidth', 2);
grid on;
title('Perfect Signal', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0.2 1 .8]);

% Add some nan's
signal([3,6,10,28]) = nan;
signal(15:20) = nan;
signal(35:41) = nan;
subplot(2,2,2);
plot(t, signal, 'bo-', 'LineWidth', 2);
grid on;
title('Actual Starting Signal with Nans', 'FontSize', fontSize);

% Get logical map of where the nans are.
nanLocations = isnan(signal)
labeledSignal = bwlabel(nanLocations)
% Find their areas
measurements = regionprops(labeledSignal, 'Area', 'PixelIdxList');
allAreas = [measurements.Area]
% Find label of those areas 5 or bigger
bigAreas = find(allAreas>=5)
% Initialize a "fixed" signal
fixedSignal = signal;
% Find indexes with that label in the original
for region = 1 : length(bigAreas)
  % Get a logical vector with the location of the nans in this stretch.
  badStretch = labeledSignal==bigAreas(region);
  % Get the t values for them
  xi = find(badStretch);
  % tack on the endpoints that we will interpolate between.
  xi = [xi(1)-1, xi, xi(end)+1];
  leftSignalValue = signal(xi(1)-1); % Left most good point.
  rightSignalValue = signal(xi(end)+1); % Right most good point.
  % Linearly interpolate over that stretch.
  fixedSignal(xi) = linspace(leftSignalValue, ...
  rightSignalValue, length(xi));
end
% Display fixed signal
subplot(2,2, 3);
plot(t, fixedSignal, 'bo-', 'LineWidth', 2);
grid on;
title('Fixed Signal', 'FontSize', fontSize);

[Roan Leeuwenburg]

Thanks, I'm going to use this to test my own interpolate function. Very helpfull :)