How to remove unwanted values from an array?

I have one array which looks like the following:

'888888888888888888880001'  -40  54  181.012758165326
'888888888888888888880002'  -41  155  246.860345748422
'888888888888888888880003'  -38  212  297.763557297433
'888888888888888888880004'  -42  335  311.493634951638
'888888888888888888880005'  -44  461  325.026011960126
'888888888888888888880006'  -45  706  328.152463876203
'888888888888888888880007'  -41  747  317.266004051673
'888888888888888888880008'  -45  941  304.192311286292
'888888888888888888880009'  -39  1084  283.163508794566
'888888888888888888880010'  -38  1258  313.845862098729
'888888888888888888880011'  -33  29  71.2819417558447
'888888888888888888880012'  -37  106  187.141362685163
'888888888888888888880013'  -31  190  205.110242402620
'888888888888888888880014'  -40  301  228.931473501683
'888888888888888888880015'  -36  427  336.367180334263
'888888888888888888880016'  -35  560  299.892770765854
'888888888888888888880017'  -35  656  292.262336479084
'888888888888888888880018'  -37  799  285.905285953032
'888888888888888888880019'  -40  966  288.908718886324
'888888888888888888880020'  -37  1060  292.824547512183
'888888888888888888880021'  -30  1209  256.883485018257

I have another array which contains the information in the attached file. A sample of a few rows for formatting are as follow:

'888888888888888888880001'  -66  9
'888888888888888888880001'  -52  10
'888888888888888888880001'  -47  20
'888888888888888888880001'  -56  26
'888888888888888888880001'  -47  38
'888888888888888888880001'  -51  42
'888888888888888888880001'  -40  47
'888888888888888888880001'  -40  54
'888888888888888888880001'  -51  62
'888888888888888888880001'  -46  80
'888888888888888888880001'  -50  85
'888888888888888888880001'  -41  94
'888888888888888888880001'  -52  98
'888888888888888888880001'  -52  109
'888888888888888888880001'  -66  141
'888888888888888888880001'  -59  143
'888888888888888888880001'  -46  144
'888888888888888888880001'  -48  150
'888888888888888888880001'  -48  159
'888888888888888888880001'  -48  166
'888888888888888888880001'  -57  167
'888888888888888888880001'  -57  170
'888888888888888888880001'  -66  199
'888888888888888888880001'  -66  204
'888888888888888888880001'  -53  205
'888888888888888888880001'  -57  218
'888888888888888888880001'  -52  224
'888888888888888888880001'  -50  272
'888888888888888888880001'  -59  296
'888888888888888888880001'  -56  303
'888888888888888888880001'  -53  350
'888888888888888888880001'  -53  371
'888888888888888888880001'  -57  380
'888888888888888888880001'  -53  448
'888888888888888888880001'  -57  464
'888888888888888888880001'  -55  620
'888888888888888888880001'  -60  861

See file for full list.

I am wanting to go through each row in the first array, and remove elements in the second array that do not match a criteria. The logic is as follows:

1) For row 1 in the first array, find the row in the second array which has the same first 3 columns.

2) Remove the elements in the second array if the third column in the second array is between the third column in the first array +- the value from the fourth column in the first array.

Example for the provided data:

First Row in the first array:

'888888888888888888880001' -40 54 181.012758165326

Only keep elements in the second array if the third column in the second array is between (54 - 181.012758165326) and (54 + 181.012758165326)....between -127 and 235. The resulting array would be:

'888888888888888888880001'  -66  9
'888888888888888888880001'  -52  10
'888888888888888888880001'  -47  20
'888888888888888888880001'  -56  26
'888888888888888888880001'  -47  38
'888888888888888888880001'  -51  42
'888888888888888888880001'  -40  47
'888888888888888888880001'  -40  54
'888888888888888888880001'  -51  62
'888888888888888888880001'  -46  80
'888888888888888888880001'  -50  85
'888888888888888888880001'  -41  94
'888888888888888888880001'  -52  98
'888888888888888888880001'  -52  109
'888888888888888888880001'  -66  141
'888888888888888888880001'  -59  143
'888888888888888888880001'  -46  144
'888888888888888888880001'  -48  150
'888888888888888888880001'  -48  159
'888888888888888888880001'  -48  166
'888888888888888888880001'  -57  167
'888888888888888888880001'  -57  170
'888888888888888888880001'  -66  199
'888888888888888888880001'  -66  204
'888888888888888888880001'  -53  205
'888888888888888888880001'  -57  218
'888888888888888888880001'  -52  224

Your help would be greatly appreciated.

[Walter Roberson]

Should

1) For row 1 in the first array, find the row in the second array which has the same first 3 columns.

read

1) For row 1 in the first array, find the row in the second array which has the same first 2 columns.

??

[Aaron]

It should be the same first 3 columns. The 3rd column will also be the same. I just want to find that row and then move up and down based on column 4 in the first array

Answer 1

[Andrei Bobrov]

f = fopen('C:\OneArray.txt');
ca = textscan(f,'%s  %f  %f %f');
fclose(f);
f = fopen('C:\Example5.txt');
data = textscan(f,'%s %f %f');
fclose(f);

lm = [ca{:,3}-ca{:,4}, ca{:,3}+ca{:,4}];
[lgc,ii] = ismember(data{1},ca{1});

idx=ii(lgc);
funa = @(x){data{3}(x) >= lm(idx(x(1)),1) & data{3}(x) <= lm(idx(x(1)),2)};
p = accumarray(idx,(1:numel(data{1}))',[],funa);
idxout = cat(1,p{:});

d1 = cellfun(@num2cell,data(2:end),'un',0);
d1 = [data{1}, cat(2,d1{:})];

out = d1(idxout,:);

[Aaron]

if true
  % code
end

I have my arrays and when I run the first line of code in that it tells me that:

Error using  - 
Too many input arguments.

Error in AutomatedResults_V3 (line 40)
lm = [out5{:,3}-out5{:,4},out5{:,3}+out5{:,4}];

My array 1 is out5 and my array with the data is sort_data. I get them in the following way:

sort_data = sortrows(matrix_data, [1,3]);
out5 = [out4(:,1),out4(:,2),out4(:,3),out3(:,3)];

Then I try to run the first line of your answer like this:

lm = [out5{:,3}-out5{:,4}, out5{:,3}+out5{:,4}];

And it gave me the error message above. I figure it has something to do with how my variables are defined but I dont know. I also tried:

lm = [out5(:,3)-out5(:,4), out5(:,3)+out5(:,4)];

But got this message:

Undefined function 'minus' for input
arguments of type 'cell'.

Error in AutomatedResults_V3 (line 40)
lm = [out5(:,3)-out5(:,4),
out5(:,3)+out5(:,4)];

Do you know what might be causing this?

[Andrei Bobrov]

What is your data 'out3' and 'out4'. What data type.

[Aaron]

I think it is cell array. out3 i get from:

data = textscan(fid, '%s %f %f');

fclose(fid);

matrix_data = [data{1}, num2cell(data{2}), num2cell(data{3})];

sort_data = sortrows(matrix_data, [1,3]);

[a,ii,ii] = unique(sort_data(:,1));
[j1,j2] = ndgrid(ii,1:2);
b = cat(1,sort_data{:,2:3});

out = [a, accumarray([j1(:),j2(:)],b,[],@(x) {max(x)})];
out2 = [a, accumarray([j1(:),j2(:)],b,[],@(x) {std(x)})];
out3 = [out(:,1),out(:,2),out2(:,3)];

out4 I got from the last question you answered. I am not very creative with my variable names.

[Walter Roberson]

lm = [ca{:,3}-ca{:,4}, ca{:,3}+ca{:,4}];

should be

lm = [ca{3}-ca{4}, ca{3}+ca{4}];

[Aaron]

That got rid of my error for that line, now I have an error that says:

Error using accumarray Second input VAL must be a vector with one element for each row in SUBS, or a scalar.

Error in AutomatedResults_V3 (line 44) p = accumarray(idx2,(1:numel(sort_data{1}))',[],funa);

The code is as follows for this section:

lm = [out5{3}-out5{4}, out5{3}+out5{4}];
[lgc,ii3] = ismember(sort_data{1},out5{1});
idx2 = ii3(lgc);
funa = @(x){sort_data{3}(x) >= lm(idx2(x(1)),1) & sort_data{3}(x) <= lm(idx2(x(1)),2)};
p = accumarray(idx2,(1:numel(sort_data{1}))',[],funa);
idxout = cat(1,p{:});
d1 = cellfun(@num2cell,sort_data(2:end),'un',0);
d1 = [sort_data{1}, cat(2,d1{:})];
outFinal = d1(idxout,:);

[Aaron]

Any ideas on how to fix this?

[Walter Roberson]

In the accumarray line, change idx2 to idx2(:)

[Aaron]

When I changed that I got:

Attempted to access lm(20,1); index out of bounds because size(lm)=[1,48].

Error in @(x){sort_data{3}(x)>=lm(idx2(x(1)),1)&sort_data{3}(x)<=lm(idx2(x(1)),2)}

Error in AutomatedResults_V3 (line 44) p = accumarray(idx2(:),(1:numel(sort_data{1}))',[],funa);

[Walter Roberson]

Andrei will have to answer that; I am not sure what is is trying to do there.

[Aaron]

I have no clue what most of this means, I am just trying to automate some results and this code makes no sense to me. I think what he posted will work, I will just have to create it as a separate function or something. Thanks for trying.