F-Test Fixed Effects SSRs - reduced model smaller than complete model

Hello! I'm trying to make a modelwide (not individual coefficient specific, like with anova) f-test for my fixed-effects regressions - I can't seem to find a native function in matlab for them so I'm just building them, but if you've got one send it my way. My plan was to do something like the following:

fe_green_money = fitlme(fixed_effects_green_data, 'LogPatents ~ MoneyInvested + Dummy0Patents + (1|Organization)')
fe_green_money_reduced = fitlme(fixed_effects_green_data, 'LogPatents ~ 1 + (1|Organization)') 

F_fe_green_money = ((fe_green_money_reduced.SSR - fe_green_money.SSR) / 2) / (fe_green_money.SSR / fe_green_money.DFE);
p_fe_green_money = 1 - fcdf(F_fe_green_money, 2, fe_green_money.DFE)

But I'm running into a problem - fe_green_money_reduced.SSR is smaller (almost 2/3rds the size) than fe_green_money.SSR, which as I understand it should be impossible... Is there an alternate way I should be getting the SSRs or am I doing something wrong or is there another explanation I'm missing?

Answer 1

[Jeff Miller]

SSR is the sum of squares explained by the model (maybe you are thinking it is the sum of squares residual, which is actually called SSE in the fitlme output?).

So, the reduced model explains less of the variance--about 2/3 in your example--exactly as expected. It would be impossible for the reduced model to have a smaller SSE, not SSR.

[Zachary]

Hi Jeff,

Thank you, this is really helpful, but this produces its own strange question - I'm finding that the sum of square residuals for combined fixed effects models is smaller than the sum for the models grouped separately (which as I understand it should be impossible...). For instance:

fe_green_money = fitlme(fixed_effects_green_data, 'LogPatents ~ MoneyInvested + Dummy0Patents + (1|Organization)')

Here, fe_green_money.SSE = 479.5070

fe_nongreen_money = fitlme(fixed_effects_nongreen_data, 'LogPatents ~ MoneyInvested + Dummy0Patents + (1|Organization)')

Here, fe_nongreen_money.SSE = 456.6005

fe_combined_money = fitlme(vertcat(fixed_effects_green_data,fixed_effects_nongreen_data), 'LogPatents ~ MoneyInvested + Dummy0Patents + (1|Organization)');

Here, fe_combined_money.SSE = 936.0977

And 936.0977 < 456.6005 + 479.5070 = 936.1075

Is this an element of fixed effects regressions that I'm just not familiar with?

[Jeff Miller]

Sorry, that does look a bit unusual but I can't explain it. Are the parameter estimates nearly identical for all three fits? It seems like they must be (and this is just a little rounding error creeping in somewhere). But it seems quite unlikely that you would get nearly identical parameter estimates for all three datasets.