visdiff, Can you include a tolerance?

I'm trying to use visdiff to compare the output of two versions of a software, and make sure that the changes I made are not altering the result. I've saved the outputs to a .mat file and have several different .mat files which I would like to compare to their corresponding previous versions.

The problem is visdiff will return variables not matching even if they are very close. I would like to find some way to compare the two files, while allowing for a tolerance. For instance, the following two numbers are pulled from corresponding .mat files from the two versions, and visdiff says they're "different"

Old Version: 84.972465581755998 New Version: 84.972465581758897

My application is not concerned with how well these match at the 12th decimal point.

Thanks for any help!

JR

Answer 1

[Sean de Wolski]

isclose = @(x,y)isequal(size(x),size(y))&&all(abs(x(:)-y(:))<10^-10)
isclose(84.97246558175599,84.972465581758897)

Edit Per Comments

x = magic(3); %sample data
y = rand(10,1);
z = 'hello world';

save('ans427.mat','x','y','z')  %save it twice
save('ans4272.mat','x','y','z')

S1 = importdata('ans427.mat');  %load the stuff
S2 = importdata('ans4272.mat');

fns1 = fieldnames(S1);  %get fields.
fns2 = fieldnames(S2);
for ii = 1:length(fns1) %compare every field
  bool = isclose(S1.(fns1{ii}),S2.(fns2{ii}));
  if ~bool
      break
  end  
end

obviously you add more checks to make this faster: are there the same number of field names? do the field names have to be the same?

This is just a simple example.

[John Robinson]

Thanks Sean,

Do you know if that can be used large scale? IE, to compare an entire .mat file, or integrate in with 'visdiff'?

JR

[Sean de Wolski]

Yes. Right now it's setup to return a scalar value, are ALL of the entries close (10^-10th) close. If you want to know which ones are or aren't you could change it to

isclose = @(x,y)abs(x-y)<(10^-10)

or similar.

[John Robinson]

Thanks again Sean,

I can see how this could be used to compare individual vectors in the .mat file, but if I wanted to compare the entirety of both .mat files to each other, it doesn't seem to work if I try something like:

isclose = @(x,y)isequal(size(x),size(y))&&all(abs(x(:)-y(:))<10^-10)

ans = isclose(MatFile1.mat,MatFile2.mat)

I'm sorry that this is so basic a question, but I can't seem to translate what you're saying into a working line of code :) (I also tried the second isclose from above)

Thanks again for the help!

[Sean de Wolski]

see edit.

[John Robinson]

Hey Sean,

You're the man! It works, I modified the code a little to allow for one file to have variable names different from the other, I've put your modified code below in case anyone else wants to use it. Thanks again!

function matCompare(file1Name, file2Name, tol)

% 2011.06.27

%

% Function which compares the contents of two '.mat' files within a

% specified tolerance (tol) and will return the name of variables which do

% not match within the said tolerance. Works on the same principle as

% visdiff, but can be done with a tolerance.

%

% Adopted from work done by MATHWORKS user 'Sean de' (aka super awesome

% helpful MATLAB user!) Original help question here:

% http://mathworks.com/matlabcentral/answers/10380-visdiff-can-you-include-a-tolerance

S1 = importdata(file1Name);

S2 = importdata(file2Name);

isclose = @(x,y)isequal(size(x),size(y))&&all(abs(x(:)-y(:))<tol);

fns1 = fieldnames(S1);

fns2 = fieldnames(S2);

for ii = 1:length(fns1)

ind2 = find(strcmp(fns1{ii},fns2));

if strcmp(fns1{ii}, fns2{ind2})

bool = isclose(S1.(fns1{ii}), S2.(fns2{ind2}));

if ~bool

disp(fns1{ii});

end

else

disp('Oh Boy!')

disp(fns1{ii})

disp(fns2{ind2})

end

end

Answer 2

[Walter Roberson]

What datatypes need to be compared? You will have trouble with objects, handle graphics, anonymous functions, or inline functions.

If the .mat files include structures, you will need to define whether the output should be considered the same if the numeric values are "close enough" but the field order is different.

If your files can include the integer datatypes such as uint16, then you will need to define what is "close enough" for them.

Is "close enough" to be defined in absolute terms, or should it be defined relatively, perhaps in multiples of eps() of the values?

If you have single precision values, how would you like to compare those?

[Walter Roberson]

Do the variable names have to be in the same order in the .mat file?

[John Robinson]

Thanks for the input Walter,

The .mat files which I am comparing have mostly vectors of real numbers (no structures which I can see).

It seems like depending on which data type there would be a lot more to consider than what we've got above, but for my case (at least now) what Sean came up with seems to work wonderfully.

Thanks again for the input, it's always appreciated!

JR