How to drop last N round-off error digits?

a=sqrt(pi) returns 1.77245385090552, where the last digit is not reliable, and this digit is of the order 1e-14.

Now, a=sqrt(53512343*pi) returns 12965.8776658265, where the last digit or two are not reliable, and these digits are of the order 1e-9 -- 1e-10.

The question is how to get rid of unreliable digits in a standardized and well-proven manner?

The problem is that these last digits may deviate strongly depending on input data of an algorithm leading to situations when some results that should be in fact equal are not equal in the last digit.

Note that

format long

should be executed before typing any expressions in order to see the full number of digits of the results.

Note also that "comparison up to some precision" is not a solution, as I need to store only one unique value of the answer, thus neglecting all its variations due to round-off errors. And I'm not aware about any unique()-like function that works with some tolerance.

The round-off errors in my application are usually small (i.e. they are in the last 1-2 digits).

Answer 1

[Titus Edelhofer]

Hi Ilya,

if you are happy with an output, then use fprintf:

a=sqrt(pi);
fprintf('a = %.14g\n', a)
a = 1.7724538509055

Otherwise: use e.g. eps to find the roundoff digit.

a = sqrt(pi);
e = eps(a);
% how many decimal digits it this?
digits = floor(-log10(e));
% cut off the last two
digits = digits - 2;
% round to this digit
aRound = round(a*10^digits) / (10^digits)
aRound =
 1.772453850905500

Titus

[Ilya]

Of course, I need the second option =) And it looks pretty nice.. However, I wonder if there's some standardized solution since the problem seems to be quite common.

[Srighakollapu Vali M]

This works fine for non zero values. If the variable takes 0 instead of 'pi' or some nonzero number, after running this code, I am getting NaN.How to deal the above problem if the value is zero especially, if we are working with matrices where some entries may be zero.

Answer 2

[John D'Errico]

The simple scale/round/unscale is the standard answer. HOWEVER, don't expect it to help you completely, because you are working in floating point arithmetic. For example...

X0 = 1.2345678901234567
X0 =
        1.23456789012346

Lets try to zero out the last few digits.

X1 = round(1e12*X0)/1e12
X1 =
          1.234567890123

While it appears that this worked, did it? I'll use my HPF tool as a simple way of extracting the exact number as MATLAB stores it. Remember that MATLAB uses a binary form,so while it might print out in decimal form, and LOOK like it is what you think, the true value as stored is not so exact, EVEN AFTER ROUNDING.

hpf(X1,55)
ans =
1.234567890122999944679804684710688889026641845703125000

Finally, while format long HELPS, it does not truly show the effect of those 53 bits that MATLAB carries in the binary form.

[Ilya]

Thanks a lot! I simply received several very elegant and unexpected solutions on this forum and hoped to receive another such solution. Thanks also for the HPF tool.

Answer 3

[Albert]

floor(1e12*X0)/1e12

this may work better than

round(1e12*X0)/1e12