Clear Filters
Clear Filters

Matlab is giving different answers for the same calculation

9 views (last 30 days)
Matlab is showing different results for the same calculation when done with direct values instead of using variables.
The calculation is: h = k*Nu/D_h
k = 0.129, Nu = 8.24, D_h = 0.0019
After running the script, the answer produced is: 562.7917
Doing the same calculation in the command window through the variables produces the same answer,
however the answer produced while doing the calculation in the command window using direct values is different.
Running " 0.129*8.24/0.0019 " in the command window produces the result: 559.4526
Matlab Script:
rho = 916; %kg/m^3
C_p = 1907; %J/kg
k = 0.129; %W/m*K
v = 1.6400e-05; %m^2/s
A_c = 1.2985e-05;
P = 0.0275;
D_h = 4*A_c/P
D_h = 0.0019
Flow_Rate = 0.000075; %m^3/s
vel = Flow_Rate/(12*A_c)
vel = 0.4813
Re = vel*D_h/v
Re = 55.4324
Nu = 8.24
Nu = 8.2400
h = (k*Nu)/D_h
h = 562.7917
Script Results:
Command Window Results:
In case its relevant, the system running Matlab is 64 bit and the version of Matlab being used is:
MATLAB Version: (R2023b) Update 7
Operating System: Microsoft Windows 10 Home Single Language Version 10.0 (Build 19045)
Java Version: Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
However, the same issue was observed on Matlab online.
The answer for this calculation from other sources is the same as the result when the direct values were used (559.4526).
Why is Matlab producing different results, and can anything be done to resolve this?

Accepted Answer

Sam Chak
Sam Chak on 30 Mar 2024
The discrepancy observed can be attributed to numerical precision errors introduced by humans. The scripted answer is the accurate one in this case. By default, the displayed answer is rounded to four decimal places. If you directly input the numerical values from the displayed answer into the formula, the computation accuracy will be compromised as it truncates beyond the displayed precision.
Sam Chak
Sam Chak on 30 Mar 2024
It's important to acknowledge that human errors are relatively common, especially when working on tasks manually and individually (one-man show), as is the case with experts as well.
Dyuman Joshi
Dyuman Joshi on 30 Mar 2024
Man I gotta change my keyboard, the keys keep getting stuck so much.

Sign in to comment.

More Answers (1)

Dyuman Joshi
Dyuman Joshi on 30 Mar 2024
Edited: Dyuman Joshi on 30 Mar 2024
That's because the value of D_h is not 0.0019.
The value stored is not the same as value displayed.
rho = 916; %kg/m^3
C_p = 1907; %J/kg
k = 0.129; %W/m*K
v = 1.6400e-05; %m^2/s
A_c = 1.2985e-05;
P = 0.0275;
%Change the format
format long
D_h = 4*A_c/P
D_h =
The value is a terminating irrational value.
Now, let's see the result of manual calculation in limited precision of floating point values-
out = 0.129*8.24/0.001888727272727
out =
which in the default (short) format is
format default
out = 562.7917
which matches with the output you obtained.


Find more on Startup and Shutdown in Help Center and File Exchange




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!