How to change training and plotting options when training a deep network?

I have Matlab R2021a and I need to modify the option ‘OutputNetwork’ from the default ('last-iteration') into 'best-validation-loss' in trainingOptions by using this command:

options = trainingOptions('OutputNetwork', 'best-validation-loss');

but this option is not found in matlab R2021a. Could you please help me?

Answer 1

[Mohammad Sami]

From matlab documentation, you can only use this option if you specified validation data as well.

https://nl.mathworks.com/help/deeplearning/ref/trainingoptions.html#bu59f0q_sep_mw_87c68427-6315-4ab2-b597-e78d9f98c079

• 'best-validation-loss' – Return the network corresponding to the training iteration with the lowest validation loss. To use this option, you must specify 'ValidationData'.

[Fatma Diab]

I have validation data and my options are:

options = trainingOptions('adam', ...

'MiniBatchSize',20, ...

'MaxEpochs',10, ...

'InitialLearnRate',0.0001, ...

'Shuffle','every-epoch', ...

'ValidationData', imdsValidation, ...

'ValidationFrequency',300, ...

'OutputNetwork', 'best-validation-loss',...

'Verbose',true, ...

'Plots','training-progress');

but this error appears:

Error using nnet.cnn.TrainingOptionsADAM (line 132)

'OutputNetwork' is not an option for solver 'adam'.

Error in trainingOptions (line 317)

opts =nnet.cnn.TrainingOptionsADAM(varargin{:});

[Mohammad Sami]

This functionality is recently introduced in Matlab R2021b.

https://www.mathworks.com/help/deeplearning/release-notes.html

Network Training: Return network with lowest validation loss

When training a neural network using the trainNetwork function, output the network with the lowest validation loss by setting the OutputNetwork name-value argument of the trainingOptions function to "best-validation-loss".

You may try using checkpoint path for this perhaps. See the available options in R2020a here

CheckpointPath — Path for saving checkpoint networks

character vector

Path where checkpoint networks are saved, specified as a character vector.

Data Types: char

https://www.mathworks.com/help/releases/R2020a/deeplearning/ref/nnet.cnn.trainingoptionsadam.html?searchHighlight=training%20options%20adam&s_tid=doc_srchtitle

For more on checkpoint see here

https://www.mathworks.com/help/deeplearning/ug/resume-training-from-a-checkpoint-network.html