Trained CNN model partitioning into two networks

Hello all,

I have fine-tuned a pretrained CNN and created a system for a definite classification problem (this system is a bit complex).

I want to apply Mobile Cloud Computing (MCC). I want to divide the computation between the user's end-device and the Cloud because I don't want to send the captured image to the Cloud and I want to reduce the computational workload of the smartphone.

I want to create two separate sub-models from a single trained CNN model. So the first model will take as an input the captured image and give as output intermediate features (End-device) and the next model will take these features as input and give the final class prediction as output (Cloud side). Thus, trained CNN = model1 + model2.

Suppose the trained CNN is mobilenetv2 and it is saved as DAGnetwork after training -> 'net'. How can split 'net' into two sub-networks that can be used for the explained purpose ? I tried to customize my trained 'net' in Deep Network Designer but the model is saved as lgraph which cannot be properly used later on.

Appreciate any king of help !!! Thank you in advance

Answer 1

[Shantanu Dixit]

Hi Andrea,

'DAGNetwork' objects are not recommended by MathWorks now, use 'dlNetwork' instead.

For giving the output of model1 (from input to block 14) as input to model 2 (from block 15 to classification output layer). You can use 'minibatchpredict' to extract the intermediate features from the first part and pass these features to the second part of the model for final classification.

Below is a brief example of how to use 'minibatchpredict' to get intermediate outputs. This approach can be adapted to your 'mobilenetv2' model by specifying the appropriate layers where you want to split the model.

layers = [ 
    imageInputLayer([28 28 1], Normalization="none", Name="input") 
    convolution2dLayer(3, 8, Padding="same", Name="conv_1") 
    reluLayer(Name="relu_1") 
    fullyConnectedLayer(10, Name="fc") 
    softmaxLayer(Name="softmax") 
    ]; 

net = dlnetwork(layers);
disp(net); 
sampleInput = rand(28, 28, 1); 
dlSampleInput = dlarray(sampleInput, 'SSCB'); 

activations_conv_layer = minibatchpredict(net,dlSampleInput,Outputs="conv_1")  %% conv_1 outputs 
activations_fc = minibatchpredict(net,dlSampleInput,Outputs="fc");    %% fc layer outputs

Refer to the following MathWorks documentation for better understanding

'dagnetwork': www.mathworks.com/help/deeplearning/ref/dagnetwork

'dlnetwork': www.mathworks.com/help/deeplearning/ref/dlnetwork

'minibatchpredict’: www.mathworks.com/help/deeplearning/ref/minibatchpredict.html

[Andrea Daou]

Thank you Shantanu Dixit for this detailed explanation!