fitnet Why test results are unbiased?

3 views (last 30 days)
Emiliano Rosso
Emiliano Rosso on 2 Apr 2015
Answered: Greg Heath on 22 May 2015
I've read form Greg Heath:
Data = Design + Non-design
Design = Train(estimate weights) + Validation(Stop training when MSEval goes thru a minimum).
Non-design = Test(Obtain UNBIASED generalization estimate of performance on unseen non-design data).
I like to know why and how test results are calculated on unbiased net despite the fact that training is made on biased network and how to obtain the same results using :
I must use :
net.biasConnect = [0; 0];
after training and before using net for new outputs?
There is a contradiction between training net and the changing made after?
What do you mean when you say in:
"The test set error is unbiased because it is completely independent of design (training and validation)." ?

Accepted Answer

Greg Heath
Greg Heath on 22 May 2015
This discussion has absolutely nothing to do with bias weights.
Theoretically, training, validation and test data are all assumed to be random samples from the same probability distribution function.
Both training and validation data are used to design the net with the goal of minimizing a performance function (such as mse (fitnet) or crossentropy (patternnet) for net performance on ANY data (seen and unseen) obtained from the same probability distribution.
Therefore, their performance cannot be used to obtain an unbiased (i.e., honest) estimate of performance on unseen data.
On the other hand, the test data is in no way involved in the design of the net. Therefore it is a valid representative of "unseen" data and it's performance is considered an unbiased estimate of net performance on unseen data.
Hope this helps.
PS Not sure why I never saw this post before.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!