When use Sequence and when Feature for Neural Networks input layer in regression problem?

Hi to all,

my problem is the following: I have 100 samples each one with dimension 3, and 100 corresponding outputs with dimension 1.

I would like to use the Deep Learning Toolbox, and in particular the trainNetwork function, for finding a function which given a certain input (of 3 elements) reproduces almost the correct corresponding single output (it should be a regression problem).

In the definition of the layers of the Neural Network, I am not sure if the input layer should be sequenceInputLayer(3) or featureInputLayer(3). Notice that my samples are not time depending: essentially, they are just 100 set of 3 random values taken between 0 and 1.

I think I need to use the featureInputLayer, but I am not sure why the sequenceInputLayer should not work.

Thanks in advanced for your help!

FP

Answer 1

[Philip Brown]

You're completely correct - you want to use featureInputLayer(3) as the input layer, and regressionOutputLayer as the output.

featureInputLayer is for use with the sort of data you have: 100 samples, each with 3 features, like a 100x3 table.You probably then want to use fullyConnectedLayers to build a "multi-layer perceptron" to learn features based on those in your data. With your "table"-like data, the order of features doesn't matter - you could swap column 2 and column 3, and that doesn't change the meaning of your data, just the order you define its features in.

You'd use sequenceInputLayer(3) when you have data which has some sort of "time ordering" - for example, a time series with measurements taken at 3 points. Text data can also be thought of as a form of time series data. Then you'd probably want to use either recurrent neural network layers like lstmLayer, or a 1D-CNN using convolution1dLayer, to model time dependence in your data. (In reality, 3 timesteps is probably too short a time series to work well). For time series data, you can't swap the order of timesteps - this changes the meaning of your data.

[Francesco Porretta]

Thanks, and sorry for delay!