MSE different in fitnlm vs. manually programmed

Hello,

I was fitting specific model on the specific data with the fitnlm function.

The output structure for this, let's call it mdl, provides except anything else beta and Coefficients.Estimate data.

But when I used I used following my own function to evaluate MSE, which looks as pure as possible:

get_MSE = @(mdlfun, tr_x, tr_y, b) mean((tr_y-mdlfun(b, tr_x)).^2);

... there was little difference between

get_MSE = @(mdlfun, tr_x, tr_y, mdl.Coefficients.Estimate)

and

mdl.MSE

.

Do you have any idea, why?

Here is my whole illustratory code:

tr_x = rand([10, 2]);
tr_y = rand([10, 1]);

get_MSE = @(mdlfun, tr_x, tr_y, b) mean((tr_y-mdlfun(b, tr_x)).^2);

modelfun = @(b, tr_x) b(1)*tr_x(:, 1).*sin(b(2)*tr_x(:,2));

mdl = fitnlm(tr_x, tr_y, modelfun, [1 1]);

disp(mdl.MSE);
beta = mdl.Coefficients.Estimate;
disp(get_MSE(modelfun, tr_x, tr_y, beta));

Thanks in advance, best,

PL

Answer 1

[Jayanti]

Hi Peter,

The slight discrepancy you are seeing is because "mdl.MSE" returned by "fitnlm" is an unbiased estimate, whereas your manual computation uses a biased estimate (dividing by n instead of n - p).

"mdl.MSE" is calculated as:

Your manual function as:

Where "RSS" is the Residual Sum of Squares, n and p is the number of observations and model parameters.

To match the behavior of "mdl.MSE", you can modify your manual "MSE" calculation as follows:

% Manually compute unbiased MSE 
beta = mdl.Coefficients.Estimate;
residuals = tr_y - modelfun(beta, tr_x);

n = length(tr_y);            
p = length(beta);            
MSE_unbiased = sum(residuals.^2) / (n - p);

When you run this modified code, you should find that "mdl.MSE" and manually calculated "MSE" are very similar.