round-off issue?

6 views (last 30 days)
Leor Greenberger
Leor Greenberger on 20 Sep 2011
I can't figure out why the floor wont come to 1752?
>> 3.501/40*20*10^3+1.5
ans =
1.752000000000000e+003
>> floor(ans)
ans =
1751

Sign in to answer this question.

Accepted Answer

Fangjun Jiang
Fangjun Jiang on 20 Sep 2011
Rounding error caused by floating point calculation.
>> 3.501/40*20*10
ans =
17.504999999999995
  2 Comments
Jan
Jan on 20 Sep 2011
sprintf('%.13f', 3.501/40*20*10^3+1.5)
>> 1751.9999999999998
sprintf('%.12f', 3.501/40*20*10^3+1.5)
>> 1752.000000000000
Fangjun Jiang
Fangjun Jiang on 20 Sep 2011
Thanks, Jan! That's even better to explain the problem!

Sign in to comment.

More Answers (0)

Categories

Find more on Image Arithmetic in Help Center and File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!