Plot only adjacent points that are close to each other

I got a good answer to my question:

https://www.mathworks.com/matlabcentral/answers/543065-find-set-of-all-adjacent-points-that-are-close-to-each-other?s_tid=prof_contriblnk

Here is my MATLAB version:

MATLAB                                                Version 9.8         (R2020a)
Communications Toolbox                                Version 7.3         (R2020a)
DSP System Toolbox                                    Version 9.10        (R2020a)
Instrument Control Toolbox                            Version 4.2         (R2020a)
LTE Toolbox                                           Version 3.3         (R2020a)
MATLAB Compiler                                       Version 8.0         (R2020a)
Parallel Computing Toolbox                            Version 7.2         (R2020a)
RF Toolbox                                            Version 3.8         (R2020a)
Signal Processing Toolbox                             Version 8.4         (R2020a)
Symbolic Math Toolbox                                 Version 8.5         (R2020a)

But I neglected to say that I use dY_idx and Y_adj_close in a subsequent function call where both arguments must have the same dimensions.

Here is my code that keeps the dimensions the same. Is there any way to improve this code to make it run faster (for large vectors)?

Y = [ 1.1  1.12    9.2    8.3  8.295  8.292     4.1  4.12    4.19];
plot(Y), hold on;

% Original 2 lines of code that loses the right-most point is:
% dY_idx = find(abs(diff(Y)) < 0.022);
% Y_adj_close = Y(dY_idx)

% Here are my 5 lines of code that saves the right-most point to keep the right-most point of each grouping.
% Can these 5 lines be reduced?
logidx = abs(diff(Y)) < 0.022;
xLog = [logidx 0] | [0 logidx];
Y_adj_close = Y( xLog );
dY_idx = 1:length(Y);
dY_idx = dY_idx( xLog );

plot(dY_idx, Y_adj_close, 'r.'), hold off

[Dhanush Bejjarapu]

Hi, since you're doing basic operations, for large vectors and datasets, it is always faster to use Parallel Computing Toolbox provided by MATLAB. There are various options like GPU computing, cloud, etc., which you can find in the documentation given and it will surely make things run faster!

https://www.mathworks.com/help/parallel-computing/

[Walter Roberson]

When you are doing basic operations on large vectors, then it is almost always faster to not use Parallel Computing Toolbox. MATLAB calculates many basic operations in high performance multi-threaded libraries, automatically using multiple threads, and it does that in a way that does not have to create multiple processes and initialize MATLAB in the processes and pass the data to the processes and do synchronization to get results back.

[Paul Hoffrichter]

@Dhanush,

FYI - You need more than Parallel Computing Toolbox to work on the cloud, or for that matter, to distribute a job over multiple workstations in a local network. I listed my available toolboxes below in response to an answer.

[Paul Hoffrichter]

@Walter,

Yes, I realized that using MATLAB built-in functions were highly optimized to take advantage of the computer architecture. I ran two different versions of the same algorithm. My version used better built-in functions. The other also had more for-loops. I compared the 16 logical cores in both versions, and saw that my version was using most of the cores at high capacity; whereas the other version showed less usage and lower capacity per core. As a result, my version ran 8x faster.

But when I changed the outmost for-loop to a parfor-loop, the other approach was able to perform much better, whereas my approach, which was already heavily multi-threaded via built-in's could only improve a little more. As a result, when using parfor, my version only ran 3.5x faster.

Below is my original code that worked reasonably well for my application. But I thought that by compensating for losing a point when using diff(), I might get better results. It was only 2 lines of code. But now I have 5 lines of code, which makes me suspect that there may be a more efficient way to reduce the lines of code. I wonder if the lines of code in the OP can be reduced.

dY_idx = find(abs(diff(Y)) < 0.022);
Y_adj_close = Y(dY_idx)

[Walter Roberson]

If you are not reusing dY_idx then do not do the find()

Y_adj_close = Y(abs(diff(Y)) < 0.022);

[Paul Hoffrichter]

From the OP: "I use dY_idx and Y_adj_close in a subsequent function call where both arguments must have the same dimensions."

Answer 1

[Tommy]

logidx = abs(diff(Y)) < 0.022;
dY_idx = find([logidx 0] | [0 logidx]);
Y_adj_close = Y( dY_idx );

Seems to me that every part of these three lines is required, but maybe it can be made even faster.

Of course number of lines alone isn't the only thing to consider. I'd imagine this would be slower:

dY_idx = find([abs(diff(Y)) < 0.022 0] | [0 abs(diff(Y)) < 0.022]);
Y_adj_close = Y( dY_idx );

[Paul Hoffrichter]

Thank you Tommy. You make it look so easy. :)

Functionally both the 3- and 2- line versions work fine. I can understand that the two-liner might be slower; but maybe MATLAB has some optimization that will surprise us. I'll check the timing and report back the results when I integrate the changes into the program. Thanks again!

[Tommy]

Happy to help!

Answer 2

[KSSV]

Have a look on knnsearch. This function gives you the nearest points/ points specififed by a certain distance for a given set of points.

[Paul Hoffrichter]

I do not have knnsearch.

MATLAB                                                Version 9.8         (R2020a)
Communications Toolbox                                Version 7.3         (R2020a)
DSP System Toolbox                                    Version 9.10        (R2020a)
Instrument Control Toolbox                            Version 4.2         (R2020a)
LTE Toolbox                                           Version 3.3         (R2020a)
MATLAB Compiler                                       Version 8.0         (R2020a)
Parallel Computing Toolbox                            Version 7.2         (R2020a)
RF Toolbox                                            Version 3.8         (R2020a)
Signal Processing Toolbox                             Version 8.4         (R2020a)
Symbolic Math Toolbox                                 Version 8.5         (R2020a)

[Paul Hoffrichter]

My original code is

dY_idx = find(abs(diff(Y)) < 0.022);
Y_adj_close = Y(dY_idx)

In the OP, to get the right-most point that I miss when using diff(), I have 5 lines of code to compute dY_idx and Y_adj_close. I wonder whether it is possible to reduce the lines of code in the OP to get the right-most point in each grouping.

[Walter Roberson]

knnsearch is from Statistics and Machine Learning Toolbox.

[Paul Hoffrichter]

I have only the toolboxes listed. I have to work with what I have.

[Walter Roberson]

The file exchange has https://www.mathworks.com/matlabcentral/fileexchange/4586-k-d-tree and several other KDTree implementations.

However, if you are working in 1D then it might not be worth it, but interp1() 'nearest' might possibly be faster (I think it calls into mex code.)

[Paul Hoffrichter]

Just to be clear, I am only interested in whether two adjacent points are within a tolerance. For example, if I have 3 adjacent data points, y1, y2, y3 with following data:

y2 is close enough to y1
y3 is not close enough to y2
y3 is close enough to y1

then outcome should be to select {y1, y2} and corresponding indices. Even though y3 is close enough to y1, it is not adjacent so it is disqualified. My recollection of kd tree alg is that all three points would be accepted, but I could be wrong. And yes, I suspect that for this 1D case, the kd tree or other general algorithms would be overkill. Based on the nature of the data, I do not want y3 to be included.

Here is the big question: Can the OP code be reduced from 5 lines lines of code to 4 or less? The test for success is whether the resultant plot is identical to the OP plot. Often, less lines of code without using for-loops are better optimized by MATLAB, but I could be wrong on that account since I am fairly new to MATLAB.