How do i normalize data in neural networks ? feauture by feature or hole data in one step?

Hi, Currently, i am building a neural network with one input, one hidden and one output layer and i am at the stage of normalization of the data. i have 5 diffrent features with different measurement units. 1 row of my data is as follows:

temperature light     humidity ........(2 features more)
25 C        300 lux   250      ........

i have 3000 lines of data . So while i am conducting my normalization, do i have to normalize them feature by feature or do i have to find the max and min in all dataset and do the calculations? Thanx for answers in advance.

[Matthew]

I have no idea how Matlab supports this, but in general feature normalization should be done feature by feature so that the network starts with equal sensitivity to all the features. In addition sometimes the architect may choose to do feature normalization based on other limits besides the max/min -i.e if you have extreme outliers that you haven't thrown away, you may not want to normalize based on them.

Answer 1

[Brendan Hamm]

Yes. Data would be normalized feature by feature as it would not make sense to divide something in units of C by something in units of lux.

If you are building this using the Neural Network Toolbox this is done automatically for you by mapping the data of each feature to the range [-1,1] using the mapminmax function. Similarly this is also done for the targets at the output layer.

That being said, if you are normalizing them 1 at a time, you can do this using vectorized functions. If you did have "outliers" in your data then the zscore function may be a more appropriate form of normalization.

[Greg Heath]

I would only use zscore to search for outliers that have to be removed or modified.

Then use the original scales and let the network's automatic scaling take over from there.

Hope this helps.

Greg