MLP using the Deep Learning Toolbox; Iteration per epoch is 1 in every epoch.

I am training a MultiLayer Perceptron using the Deep Learning Toolbox. I have specified the size of Mini Batch training. However, while training on every epoch, the model trains through the entire dataset once and does not iterate over the different batches of data.

This is the code.

% Network Architure
networkLayers = [sequenceInputLayer(1122,'Name','Input')
       fullyConnectedLayer(750,'Name','Hidden')
       reluLayer('Name','ReLU-Activation1')
       dropoutLayer(0.4,'Name','dropout_Regularization')
       fullyConnectedLayer(1,'Name','Output')
       reluLayer('Name','ReLU-Activation2')
       regressionLayer('Name','RegressionOutput')];

% Parameter setting       

XValidation = features(:, 80:99);
YValidation = target(:, 80:99);

maxEpochs = 60;
miniBatchSize = 20;
validationFrequency = floor(numel(target)/miniBatchSize);

options = trainingOptions('adam', ...
    'MaxEpochs',maxEpochs, ...
    'MiniBatchSize',miniBatchSize, ...
    'InitialLearnRate',0.01, ...
    'GradientThreshold',1, ...
    'Shuffle','never', ...
    'Plots','training-progress',...
    'Verbose',0);

net=trainNetwork(features,target,networkLayers,options);

Answer 1

[Nomit Jangid]

Hi Ritu,

If your input data is in a M x N matrix format (where, M = number of parameters and N = number of observations ) MATLAB assumes that this is a single observation problem with M time-series each being N points long. For each epoch, we have only 1 iteration and so the mini-batch size option is ignored because it doesn't apply to just 1 observation.

If you'd like to break the time-series into smaller chuncks of data that are treated as different observations, you can do that using the sequenceLength parameter in trainingOptions, by providing a positive integer with the desired sequence length:

https://uk.mathworks.com/help/deeplearning/ref/trainingoptions.html#bu59f0q_sep_mw_284697ea-2219-4c2f-a036-3024e6f2f313

I hope this helps.

[Ritu Panda]

Thank you so much. I tried to resolve it by the using the adamupdate function from the DL toolbox. I felt like it gave me more control over the execution of the program. But I will look into this solution as well.

[Joachim Greck]

Hello,

I have encountered the same issue and modifying the Sequence Length did solve the problem. Nevertheless, do you know a way to pre-condition my training data in a way that Matlab will automatically see it as a multiple observation problem ?

Thank you for your help,

Regards