How to save the best model during neural network training?

During the NN training there is multiple validation, in some of the epoch the validation accuracy is high. However, maybe due to the overfitting the val accuracy drops after more training. How do I save the model which have the best validation accuracy?

[KSSV]

How you are training it?

[wanting wang]

layers = [ ...
    sequenceInputLayer(inputSize)
    bilstmLayer(numHiddenUnits,'OutputMode','last')
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer];
maxEpochs = 200;
miniBatchSize = 20;

options = trainingOptions('adam', ...
    'ExecutionEnvironment','cpu', ...
    'GradientThreshold',1, ...
    'MaxEpochs',maxEpochs, ...
    'ValidationData',{XVal,YVal}, ...
    'ValidationFrequency',30, ...
    'MiniBatchSize',miniBatchSize, ...
    'SequenceLength','longest', ...
    'Shuffle','every-epoch', ...
    'Verbose',0, ...
    'Plots','none');
[net,info] = trainNetwork(XTrain,YTrain,layers,options);

That is how I construct my neural network. It end up with a output net trained after 200epoch. However, sometimes I got an ideal model with fine training accracy and high validation accuracy at around 150 or 160 epochs, I want to save that model rather than 200-epoch overfitting model.

Answer 1

[Antoni Woss]

You can choose to return the network with the optimal validation accuracy by specifying the 'OutputNetwork' name-value argument with the value 'best-validation-loss'. This will return the network corresponding to the training iteration with the lowest validation loss.

For more information on the validation options, take a look at the following documentation page: https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html