How to make predictions using an already-trained LSTM model?

Hello everyone,

I have the attached example LSTM code with the data file (omni.txt: hourly data).

I would like to know how to use the trained LSTM model to make a prediction for new data.

I think the answer lies within the lines starting from line 113, but I'm a novice with LSTM.

----------------------------------------------------------------------------------------------------------------------------------

A side question:

This code is dealing with only one input (feature) to predict its own evolution with time, How can we transform it to deal with several inputs at once?

How can we transform this code to take several inputs and predict the temporal evolution of another output?

For instance, like the feedforward backpropagation network in which it can take several inputs to predict a single output (or several outputs).

I appreciate your help!

Thank you,

Answer 1

[Hiro Yoshino]

It seems you are predicting the data based on the training data.

That's why the prediction stays unchange after the end of training data.

If you want to make predictions following the test data, you should take either way

1. feed the test data up to x_{t-1} to predict x_t
2. feed the predicted test data up to x_{t-1} to predict x_t

you may want to make a referenct to

https://jp.mathworks.com/help/deeplearning/examples/time-series-forecasting-using-deep-learning.html

Good luck!

[Mohamed Nedal]

I already feed the test data up tp X_(t-1) to predict X_(t), then I feed that predicted value to predict the next one, and so on.

%% Forecast Future Time Steps 
net = predictAndUpdateState(net, XValidation'); 
[net, YPred] = predictAndUpdateState(net, YValidation(end)); 

numTimeStepsTest = numel(XValidation); 
for i = 2:numTimeStepsTest
    [net, YPred(i)] = predictAndUpdateState(net, YPred(i-1),...
        'ExecutionEnvironment','auto'); 
end 

However, I get the same behaviour ..

[Hiro Yoshino]

I guess you're doing right.

Why don't you test the model using the training data first to see how good it is.

by the way, I wonder how many pairs of data (x, y) do you have to train the model?

I'll try to take a look at your code when I have free time.

[Mohamed Nedal]

The dataset has 16,071 pairs of (x,y), that means 16,071 days.

Ok great, that would be much appreciated. Thanks!

Answer 2

[Hiro Yoshino]

I took a look at your script.

in the line 131, you actually update the network together with getting the prediction out of it:

 [net,YPred(:,i)] = predictAndUpdateState(net,XTest(:,i),'ExecutionEnvironment','cpu'); 

[Mohamed Nedal]

Hi, sorry for my late response.

Yes but it can only predict data with the same size of the XTest.

I was wondering, after the training phase, how can I use the model to make a long-term prediction (much longer than that of the XTest).

[Hiro Yoshino]

I have taken a further look at your code.

I suppose the way you trained the network was not right.

This is a regression problem, isn't it?

So you should have

Xtrain [numOfDataset, LengthOfPeriodYouTakeIntoAccount]

Ytrain [numOfDataset, 1]

in cell format.

You can leave the imput dimension as what it is.

[Hiro Yoshino]

I have changed some lines as follows:

%% Prepare Predictors and Responses 
%XTrain = dataTrainStandardized(1:end-1); 
%YTrain = dataTrainStandardized(2:end); 
[XTrain, YTrain] = timeSeriesDataForLSTM(dataTrainStandardized, 30);

and

%% Define LSTM Network Architecture 
% Specify the LSTM layer to have 200 hidden units 
numFeatures = 1; 
numResponses = 1; 
numHiddenUnits = 100; 
layers = [ ...
    sequenceInputLayer(numFeatures)
    lstmLayer(numHiddenUnits,'OutputMode', 'last')
    fullyConnectedLayer(numResponses)
    regressionLayer]; 

The function I created for this purpose is attached. Please find it.

[Mohamed Nedal]

I got this error after updating the code (kindly check the attached code):

Error using
DAGNetwork/predictRNN>iAssertInitialStateIsValidForPredict
(line 72)
Incorrect network state. The network expects mini-batches size
of 41, but was passed a mini-batch of size 1.

Error in DAGNetwork/predictRNN (line 11)
iAssertInitialStateIsValidForPredict(statefulLayers,
dispatcher.MiniBatchSize)

Error in DAGNetwork/predictAndUpdateState (line 117)
[Y, finalState, predictNetwork] = this.predictRNN(X,
dispatcher, ...

Error in SeriesNetwork/predictAndUpdateState (line 376)
            [this.UnderlyingDAGNetwork, Y] =
            this.UnderlyingDAGNetwork.predictAndUpdateState(X,
            varargin{:});

Error in lstm (line 83)
[net,YPred] = predictAndUpdateState(net,YTrain(end));

Can you please tell me how to fix it?

[Hiro Yoshino]

Do you understand the solution I suggested? Please check out the function and have a good understanding.

Then, you should change the data accordingly after the line 64.

I got it work until 63 by changing the data format.

[Mohamed Nedal]

Hi, in the function timeSeriesDataForLSTM I wonder what n and w stand for?

[Hiro Yoshino]

w: window size

n: length of the signal

You're effectively creating a set of dataset by chopping the signal using the window size. So if the length of the signal is shorter than the window size, it won't work.

[Mohamed Nedal]

So 'window size' here referes to the batch size, right?

Kindly find the attached code with the data file.

I've used your function also for the testing set at line 53 but, yet I got the following error at line 54 in the code:

Error using
DAGNetwork/predictRNN>iAssertInitialStateIsValidForPredict
(line 72)
Incorrect network state. The network expects
mini-batches size of 31, but was passed a mini-batch of
size 128.

Error in DAGNetwork/predictRNN (line 11)
iAssertInitialStateIsValidForPredict(statefulLayers,
dispatcher.MiniBatchSize)

Error in DAGNetwork/predictAndUpdateState (line 117)
[Y, finalState, predictNetwork] = this.predictRNN(X,
dispatcher, ...

Error in SeriesNetwork/predictAndUpdateState (line 376)
            [this.UnderlyingDAGNetwork, Y] =
            this.UnderlyingDAGNetwork.predictAndUpdateState(X,
            varargin{:}); 

Excuse me for my silly questions, what should I change to make it works? And what should I change to forecast for any (n) time steps in the future?

Thank you!

[Hiro Yoshino]

No, it is a window size with which you chop the signal into peices.

you may want to take a look at this and understand how it works:

https://jp.mathworks.com/help/deeplearning/examples/time-series-forecasting-using-deep-learning.html

[Mohamed Nedal]

Hello Mr. Yoshino, I hope you're fine

I finally managed to make forecasting on new data, but the forecasting of the test data seems very inaccurate.

Kindly find the attached code with the dataset.

Any suggestions to improve the accuracy please?

Answer 3

[NGR MNFD]

Hello . I hope you have a good day. I sent the article to your service. I implemented the coding part in the MATLAB software, but to implement my network, two lines of setlayers, training MATLAB 2014 give me an error. What other function do you think I should replace? Do you think the codes I wrote are correct?( I used gait-in-neurodegenerative-disease-database in physionet website.) Thanks a lot