I am not able to use a trained neural network ?

I have a set of data :

Input to the network for training : 2 inputs

input1 1e13 1.2e13 1.23e13 ..... input2 1e09 1.01e09 1.2e09 ......

ouput 1e11 2e11 2.2e11.......

I have such 31 data points and 31 target data points,

What, I have done is used the first 25 data points and trained and tested the network using nftool, remaing 6 data points I have used the trained network to predict and selected that network which gave me the least error in the output.

My question is :

(a) The trained network shows excellent match for the target data points but as soon as I put another set of data i.e. say input1 5e13 input2 2e09 , it gives me an output which does not make sense, i.e. gives me a value lesser than that which it had given when I was training the network i.e. say input1 4e13 input2 1.8e09 , it gave a value of say 4e11 , but for the former inputs it is giving me a value of 3.8e11 , which does not make any sense , How do I train the network better so that at least it follows the trend i.e. when input1 increases the output or the value thrown by the network should increase ?

(b) I even tried normalizing the data using premnmx, but not sure how do I use the trained network with a single value say input1 5e13 input2 2e09 i.e. How do I normalize this input, what shoudl be the reference to normalize the same ?

Your inputs would help tremendously,

Thanks, Tonu

Answer 1

[Greg Heath]

 [ I N ] = size(x)  % [ 2 31 ]
 [O N  ]= size(t)% [ 1 31]

1. For precision computing transform to zero mean and unit biased variance data (zx = zscore(x',1)', etc)

2. First time through use as many defaults as possible (trainlm with trn/val/tst =21/5/5 or trainbr with /25/0/6)

> What, I have done is used the first 25 data points and trained and tested > the network using nftool, remaing 6 data points I have used the trained > network to predict and selected that network which gave me the least > error in the output.

Very insufficient detail. Network type? No. of hidden nodes? No. of candidate designs with different initial weights? How did you avoid overfitting the network and memorizing training data?(val or trainbr?)

Many designs are needed to eliminate unfortunate values of initial weights and make sure that the network is not overfit (more weights than than the 26 training equations) and overtrained (memorizing training data without val or trainbr)).

First practice with MATLAB datasets to make sure you have enough data to obtain a good design.

Next use as many defaults as possible.

If no validation set use trainbr instead of trainlm.

If you are not using trainbr, try to minimize the number of hidden nodes.

Next time you post provide more details

Hope this helps.

Thank you for formally accepting my answer

Greg

[Image Analyst]

Tonu's "Answer" moved here:

Very insufficient detail. Network type? No. of hidden nodes? No. of candidate designs with different initial weights? How did you avoid overfitting the network and memorizing training data?(val or trainbr?)

(a) i am using the default values in trainlm, No. of hidden modes I am varying between 5-25.

(b) Can you explain me how to avoid overtraining ?

The problem I am facing now is that for the training set the network gives the good results , but when I use the trained network for unknown inputs it is giving absurd output.

I am a novice in this field. Your help would be highly appreciated.

Thanks, Tonu

[Image Analyst]

Greg's comments moved here:

STILL insufficient detail. Failing to generalize to unseen data is a symptom of overtraining an overfit net with more unknown weights, Nw, than training equations Ntrneq. Assuming, of course, the new data can be assumed to come from the same probability distribution function ( e.g., mean, variance & correlation coefficient) as the training data. I have posted many discussions on overtraining.

Short answer: If statistics of new data are similar to those of the training data use a validation set or use TRAINBR.

Long Answer:

 1. Search the comp.ai.neural-nets FAQ for the discussion of overfitting.
 2. Search CANN, MATLAB NEWSGROUP and ANSWERS for my posts using "greg" and subsets of

 overfitting overtraining Nw Ntrneq

Why don't you make the corrections and post your commented code.

Greg

[Tonu]

Apologies for the late reply ,

These are my codes for creating neural network ,

My data set is linear i.e. as inputs increase , output increases.

clear all

close all

load norm_data % Loading the dataset

matrix = norm_data; matrix = matrix';

inputs = matrix(1:2,1:34); % 2 inputs , 34 data sets

targets = matrix(3,1:34); % 1 output for each two inputs, 34 data sets

% Create a Fitting Network

hiddenLayerSize = 3;

net = fitnet(hiddenLayerSize);

% Setup Division of Data for Training, Validation, Testing

net.divideParam.trainRatio = 70/100;

net.divideParam.valRatio = 15/100;

net.divideParam.testRatio = 15/100;

net.trainParam.lr=0.4;%(learning rate)

net.trainParam.mc=0.04;%(momentum)

% Train the Network

[net,tr] = train(net,inputs,targets);

% Test the Network

outputs = net(inputs);

errors = gsubtract(targets,outputs);

performance = perform(net,targets,outputs);

o1 = outputs;

o3= sim(net,matrix(1:2,35:41)); % Using the devloped net (with 34 datasets) to predict for unknown inputs

y_exp = matrix(3,35:41); % Output from the network

    obj_fun = sum(((o3 - y_exp)./y_exp).^2) ; % comparing the model results and the actual values

    % choosing the network which gives the best results by putting if

    % condition

if obj_fun < 0.02

    save new_try_norm_data.mat

    figure(1)

     plot(1:length(o3),o3,'*',1:length(o3),y_exp,'o');

% view(net)

figure(2)

plot(1:length(o1),o1,'*',1:length(o1),targets,'o');

o4 = sim(net, matrix2');

check1 = [matrix2 o4'];

else

    try1()

end

[Greg Heath]

IT WOULD REALLY HELP IF YOU RAN YOUR CODE ON ONE OF THE EXAMPLE DATASETS IN

 help nndatasets
 doc nndatasets

>> net = fitnet(3) % NO SEMICOLON

%yields

       performFcn: 'mse'
       trainFcn: 'trainlm'
       trainParam: .showWindow, .showCommandLine, .show, .epochs,
                    .time, .goal, .min_grad, .max_fail, .mu, .mu_dec,
                    .mu_inc, .mu_max

%Therefore lr and mc are not defined for the default trainlm.

% Since mse is the performance function

 obj_fun = mse(o3-y_exp)

%Proper normalization by the variance of the target yields

 normobj_fun = obj_fun/var(y_exp,1)

% The coefficient of determination, ( or R-squared See Wikipedia r-squared)is

 R2 = 1-normobj_fun

% R2 > 0.99 means the net is sucessfully accounting for 99% of the target variance.