What is the parameter minimum performance gradient (trainParam.min_grad) of traingd?

I use the training function "traingd" to train a shallow neural network:

trainedNet = train(net,X,T)

For the training function "traingd": How is the parameter minimum performance gradient (net.trainParam.min_grad) defined?

As the gradient for the gradient descent is usually a vector, but net.trainParam.min_grad is a scalar value, I am confused.

Is it the change in the performace (loss) between 2 iterations, and if yes: Does it refer to the training, validation or testing errror?

Thanks in advance!

I use MATLAB 2013 and 2015 with the neural network toolbox.

Answer 1

[Rishabh Mishra]

Hi,

Based on your description of the issue, I would state a few points:

1. I agree that gradient descent is vector quantity & points in the direction of maximum change of the cost function.
2. The ‘net.trainParam.min_grad’ is a scalar(numeric) quantity. The parameter ‘min_grad’ denotes the minimum magnitude (which is scalar) of gradient descent (which is vector), for which the training of neural network terminates.
3. When the magnitude of gradient descent becomes less than ‘min_grad’, the neural network model is said to be optimized (and hence, further training stops).

For better understanding, refer the following links:

• solving problem for gradient descent
• 'traingd' function

Hope this helps.

[AntonyH]

Thank you for this perfect answer!

[Mohamed Elsefy]

How to get the gradient value at which the trained is stopped?