Prediction ability of a neural network

Hi all,

recently I am doing some simulation and prediction work using neural networks. During the days, I have some theoretical questions regarding to NN. I appreciate it so much if anyone can help me solve them:

1. Is there any relationship between a neural network and a finite-state machine?

From my point of view, I always feel that the training of a neural network is just the process to teach the network to know its input-output relations. Sometimes, if you give a input u_i within the input range [u_min, u_max] but never occurs in the trained data, the reliability of the output is really poor.

2. How to make sure that the prediction of a neural network is reliable?

3. Somebody told me that, if a problem can be solved by traditional methods, never use neural networks. Therefore, I am wondering that, besides the high computation complexity, does neural networks have any other drawback or limitation?

Thank you very much for the help!

Answer 1

[Greg Heath]

1. Probably (I've forgotten the definition of a finite state machine, but the NN is a universal approximator)

2. The net is not a mind reader. It learns via samples from representative I/O examples. As far as it is concerned, [ umin, umax ] is determined by training data. Remember, the default neural net approximation is a linear superposition of bounded tanh functions.

3. The three largest problems with neural networks:

   a. They cannot read minds.
   b. They cannot program themselves.
   c. They must rely on humans.

HTH

Thank you for formally accepting my answer

Greg

[Yiming]

Hi Greg,

thanks for your reply.

1. I am thinking of finite-state machine is because that I've been wondering how a NN deals with a new input. Let's assume that I have only one input and the input space is finite like u = [u1,u2,u3,u4,u5] and it follows that u1<u2<u3<u4<u5. If I just train the network with a data set concluding u1,u2,u4,u5, how does the NN deal with a input u3? Is this output reliable? What about a multiple-input case?

2. Do you think if I manually convert both input and output spaces into finite will improve the performance of this NN? For instance, if I have both continuous input and output spaces as [0 10] and [10 20], I can manually convert all inputs and outputs into integers, then the input and output spaces will become [0 1 2 3 4 5 6 7 8 9 10] and [10 11 12 13 14 15 16 17 18 19 20]. Do you think it help?

Right now I am really messed up by NN and some other nonlinear system identification stuff. I hope these questions are not too stupid.

Thank you so much for your help!

Yiming

[Greg Heath]

1. I would have much more confidence in the result from input u3 than the result from either u0 or u6. There is no guarantee and should be no expectation that a net that interpolates well in the subspace spanned by the training data will extrapolate well outside of that subspace.

The exception, of course is a well trained time-series with well established significant output auto or output/input cross correlations.

2. The ranges of the input and output spaces for a NN are defined by the training data, not the full space from which the training data was obtained.

3. The best way to answer your questions is to try a test case and, if there are questions, post code, comments and error messages.

HTH