Neural Network Time Series Prediction - changing the inital state

Hello, I'm working currently with prediction-problems for dynamical systems, e.g. single pendulum with friction. At the moment I'm testing neural networks for time series predictions, although my knowledge is very basic. My understanding of neural networks in light of dynamical systems is that they are working like a flexible state-space-model. Training the neural network with some testdata should result in an accurate state-space-model, which can be used for predictions, am I right?

Lets say, I split my testdata into two sections. The first one will be used for training purpose and the second one for validation (in reference to my attached file). The prediction gives good results on the validation data, going forward, we are using the same net, but vary the inital state, here the inital angle of the pendulum. Is it even possible to vary the inital state? Does the net just predict on a one-off basis of the training data?

Regards, Michael

Answer 1

[Greg Heath]

You are using inappropriate terminology for NNs:

Neural network data division is threefold: training, validation and testing:

 total = design + test

 design = train + validate

 non-design = test

 non-training = validate + test

TRAINING: Given input(x)/target(t) training data and parameters, estimate the network weights and biases to minimize the error between the output(y) and the target(t).

Training error calculations are BIASED because the data used to calculate the error is the same data that is used to estimate the weights. If the number of estimated weights, Nw, is smaller than the number of training equations, Ntrneq, then the bias can be reduced by calculating the number of degrees of freedom, Ndof that result after the Nw weights are estimated using the Ntrneq training equations:

Ndof = Ntrneq - Nw

If the error and sum-squared error of the training data are given by

etrn = ttrn - ytrn;          SSEtrn = sum(  etrn.^2),

then the biased and degree-of-freedom-adjusted estimates of the training data mean-squared-error are

MSEtrn = SSEtrn/Ntrneq       and      MSEtrna = SSEtrn/Ndof

VALIDATION: Although validation data is used for design, it is not used to to directly estimate weights. It's purpose is to make sure that the net will perform well on non-training data. Therefore it's error is continuously monitored during training. If the error monotonically increases for a specified number of epochs (e.g., the default is 6) , then the training is stopped.

Although validation is part of the design process, the error estimates for the validation data are only SLIGHTLY biased. Therefore, no effort is made to estimate the bias or adjust the estimate.

TESTING: Although the test subset error is also continuously calculated during training, it is in no way involved in design. Therefore it's error is an UNBIASED estimate of the networks performance on non-training data (e.g., validation, testing and unseen).

Although MATLAB NN functions automatically use random data-division in the trn/val/tst proportion 0.7/0.15/0.15. For time-series I usually advise the use of the data-division option 'divideblock' that retains the data's time sequential order with the validation data placed between the training and test data.

Please look up some of my time-series posts using "greg" with TIMEDELAYNET, NARNET OR NARXNET. However, first look up the help and doc documentation. For example:

 help narxnet

 doc narxnet

Hope this helps.

Greg

Answer 2

[Michael]

Hello Greg,

thanks for your answer, you are right, I'm using the wrong terminology. Actually I'm farmiliar with the things you said, in my case "testing data" means the data that is used for NN creation, including training, testing and validation (in the right terminology). On the other hand "validation" is actually the prediction part of the NN. I'm referring to NN Examples ,especially example 9 (Prediction of chaotic time series with NAR neural network), there they use the same definitions.

Regards, Michael

Edit: Link works now, Paper is called: Neural Networks: MATLAB examples by Primoz Potocnik, University of Ljubjana

[Greg Heath]

WHEN I CLICK ON THE LINK I GET

404. That’s an error.

The requested URL /http%3A%2F%2Flab.fs.uni-lj.si%2Flasin%2Fwp%2FIMIT_files%2Fneural%2FNN-examples.pdf was not found on this server. That’s all we know

[Michael]

Hello Greg,

Link is now working, finally. Did you see my attached M-file myPrediction ?