combine 2 neural network with different inputs

13 views (last 30 days)
shlomo odem
shlomo odem on 14 Nov 2021
Answered: Srivardhan Gadila on 16 Nov 2021
I have 2 types of input that I need to classify together. Image and numerical information.
I need to transfer the image through CNN for example Squeeze Net and then enter the numeric information (array) and make a classification on everything.
I can not train one network for the images and then connect it to the numeric information because I have no information about the classification of the images, I only have information about the images and the numerical information together.
I tried to use deep network designer, to do something like this:
But there is an error that says that these are 2 different types of data ...
How can I make a network that receives two different types of inputs?

Answers (1)

Srivardhan Gadila
Srivardhan Gadila on 16 Nov 2021
I think as of R2021a, it is not possible to have different input layers for multiple inputs. But from R2021b I think it is possible. But again the output of featureInputLayer would be a 'CB' data whereas the output of pooling layer would be 'SSCB'.
So you will have to insert a custom layer which either removes the 'SS' dimension from the ouptut of pooling layer or expand the output of featureInputLayer to 'SSCB' (Refer to this page: dimension labels). Alternatively you can use the imageInputLayer itself instead of using featureInputLayer and resize the numerical training data accordingly by adding extra dimensions (like 1x1xnumFeatures).
Other way to work on this is to use the dlnetwork and custom training loops based workflow where you don't need any input layers.
In addition to the hyperlinks, refer to the following pages for more information: Multiple-Input and Multiple-Output Networks, Train Network Using Custom Training Loop.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!