importNetworkFromTensorFlow

Import TensorFlow network as MATLAB network

Since R2023b

collapse all in page

    Syntax

    net = importNetworkFromTensorFlow(modelFolder)
    net = importNetworkFromTensorFlow(modelFolder,Name=Value)

    Description

    net = importNetworkFromTensorFlow(modelFolder) imports a pretrained TensorFlow™ network from the folder modelFolder, which contains the network in the SavedModel format. The network comprises the layers defined in the saved_model file with the .pb extension and the learned weights in the variables subfolder. The function returns the network net as an initialized dlnetwork object.

    importNetworkFromTensorFlow requires the Deep Learning Toolbox™ Converter for TensorFlow Models support package. If this support package is not installed, then importNetworkFromTensorFlow provides a download link.

    Note

    The importNetworkFromTensorFlow function can generate a custom layer when importing a TensorFlow layer. For more information, see Algorithms. The function saves the generated custom layers in the +modelFolder namespace.

    example

    net = importNetworkFromTensorFlow(modelFolder,Name=Value) specifies additional options using one or more name-value arguments. For example, you can specify a namespace in which to save the generated custom layers and associated functions.

    example

    Examples

    collapse all

    This example uses:

    Open Live Script

    Import a pretrained TensorFlow network in the SavedModel format and use the imported network to predict class labels.

    Specify the model folder.

    if ~exist('digitsdlnetwork','dir')
    unzip('digitsdlnetwork.zip')
end
modelFolder = './digitsdlnetwork';

    Specify the class names.

    classNames = {'0','1','2','3','4','5','6','7','8','9'};

    Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object.

    net = importNetworkFromTensorFlow(modelFolder)
    Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

    net = 
  dlnetwork with properties:

         Layers: [12×1 nnet.cnn.layer.Layer]
    Connections: [12×2 table]
     Learnables: [6×3 table]
          State: [0×3 table]
     InputNames: {'input_1'}
    OutputNames: {'activation_1'}
    Initialized: 1

  View summary with summary.

    Read the image you want to classify and display the size of the image. The image is a grayscale (one-channel) image with a size of 28-by-28 pixels.

    digitDatasetPath = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset');
I = imread(fullfile(digitDatasetPath,'5','image4009.png'));
size(I)
    ans = 1×2

    28    28

    Display the input size of the network. In this case, the image size matches the network input size. If the sizes do not match, you must resize the image using this command: imresize(I,netInputSize(1:2)).

    netInputSize = net.Layers(1).InputSize
    netInputSize = 1×3

    28    28     1

    Convert the image to a dlarray object. Format the images with the dimensions 'SSCB' (spatial, spatial, channel, batch). In this case, the batch size is 1 and you can omit it ('SSC').

    I_dlarray = dlarray(single(I),'SSC');

    Classify the sample image and find the predicted label.

    prob = predict(net,I_dlarray);
[~,label] = max(prob);

    Display the image and the classification result.

    imshow(I)
title(['Classification result ' classNames{label}])

    Figure contains an axes object. The hidden axes object with title Classification result 5 contains an object of type image.

    This example uses:

    Open Live Script

    Import a pretrained TensorFlow network in the SavedModel format. The network contains layers that the software cannot convert to built-in MATLAB layers. The software automatically generates custom layers when you import this network.

    This example uses the findCustomLayers helper function. To view the code for this function, see Helper Function.

    Specify the model folder.

    if ~exist('digitsdlnetworkwithnoise','dir')
    unzip('digitsdlnetworkwithnoise.zip')
end
modelFolder = './digitsdlnetworkwithnoise';

    Import a TensorFlow network in the SavedModel format.

    net = importNetworkFromTensorFlow(modelFolder)
    Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

    net = 
  dlnetwork with properties:

         Layers: [14×1 nnet.cnn.layer.Layer]
    Connections: [14×2 table]
     Learnables: [6×3 table]
          State: [0×3 table]
     InputNames: {'input_1'}
    OutputNames: {'activation_1'}
    Initialized: 1

  View summary with summary.

    The imported network contains layers the software cannot convert to built-in MATLAB layers, so importNetworkFromTensorFlow automatically generates custom layers in place of these layers. The function saves each generated custom layer to a separate M file in the +digitsdlnetworkwithnoise namespace in the current folder.

    Find the indices of the automatically generated custom layers using the helper function findCustomLayers, and display the custom layers.

    ind = findCustomLayers(net.Layers,'+digitsdlnetworkwithnoise');
net.Layers(ind)
    ans = 
  3×1 Layer array with layers:

     1   'concatenate_1'      Concatenate     digitsdlnetworkwithnoise.kConcatenate1Layer3826
     2   'gaussian_noise_1'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise1Layer3766
     3   'gaussian_noise_2'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise2Layer3791

    Plot the network architecture.

    plot(net)
title('Network Architecture')

    Helper Function

    This section defines the findCustomLayers helper function. findCustomLayers returns the indices of the custom layers that importNetworkFromTensorFlow automatically generates.

    function indices = findCustomLayers(layers,Namespace)

s = what(['.' filesep Namespace]);

indices = zeros(1,length(s.m));
for i = 1:length(layers)
    for j = 1:length(s.m)
        if strcmpi(class(layers(i)),[Namespace(2:end) '.' s.m{j}(1:end-2)])
            indices(j) = i;
        end
    end
end

end

    This example uses:

    Open Live Script

    Import a pretrained TensorFlow network in the SavedModel format, which results in an uninitialized network. Then, initialize the network.

    Specify the model folder.

    if ~exist("noInputLayerTFModel","dir")
    unzip("noInputLayerTFModel.zip")
end
modelFolder = "./noInputLayerTFModel";

    Import the model using the importNetworkFromTensorFlow function. Because this model does not have a fixed input size, the function imports the model as an uninitialized dlnetwork object without an input layer. The software displays a warning that describes how to initialize the network.

    net = importNetworkFromTensorFlow(modelFolder)
    Importing the saved model...
Translating the model, this may take a few minutes...

    Warning: Size of the input to "input_5d" is unknown in TensorFlow.

    Finished translation. Assembling network...

    Warning: Network is imported as an uninitialized dlnetwork object. Before using the network, add input layer(s):

inputLayer1 = sequenceInputLayer(<inputSize1>, Normalization="none");
net = addInputLayer(net, inputLayer1, Initialize=true);

For more information on which format(s) to use, see documentation for dlarray.

    Import finished.

    net = 
  dlnetwork with properties:

         Layers: [5×1 nnet.cnn.layer.Layer]
    Connections: [4×2 table]
     Learnables: [4×3 table]
          State: [0×3 table]
     InputNames: {'conv2d_20'}
    OutputNames: {'dense_12_softmax'}
    Initialized: 0

  View summary with summary.

    Specify the input size of the imported network and create an sequence input layer. Then, add the sequence input layer to the imported network and initialize the network using the addInputLayer function.

    inputLayer1 = sequenceInputLayer([28 28 3], Normalization="none");
net = addInputLayer(net,inputLayer1,Initialize=true)
    net = 
  dlnetwork with properties:

         Layers: [6×1 nnet.cnn.layer.Layer]
    Connections: [5×2 table]
     Learnables: [4×3 table]
          State: [0×3 table]
     InputNames: {'sequenceinput'}
    OutputNames: {'dense_12_softmax'}
    Initialized: 1

  View summary with summary.

    The network contains the input layer and is ready to use for prediction. You can analyze the network further using the Deep Network Designer app. 

    deepNetworkDesigner(net)
    Open Live Script

    Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object. Analyze the network using the analyzeNetwork function.

    net = importNetworkFromTensorFlow("CustomConvReluBN");
    Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation. Assembling network...
Import finished.

    analyzeNetwork(net)

    default.png

    net is a dlnetwork object with seven layers. The second and third layers are networkLayer objects that each contain a nested neural network.

    Expand the networkLayer layer using the expandLayers function and analyze the network.

    netExpanded = expandLayers(net);
analyzeNetwork(netExpanded)

    defaultexpanded.png

    The app shows that the expanded network netExpanded has four more layers than net.

    Import the TensorFlow network again, using custom layers to represent the nested neural network. Analyze the network.

    netCustom = importNetworkFromTensorFlow("CustomConvReluBN",PreferredNestingType="customlayer");
    Importing the saved model...
Translating the model, this may take a few minutes...

    Warning: The namespace "CustomConvReluBN" already exists in the Current Folder, and will be overwritten.

    Finished translation. Assembling network...
Import finished.

    analyzeNetwork(netCustom)

    custom.png

    netCustom contains seven layers. The second and third layers are custom layers that represent nested neural networks.

    Input Arguments

    collapse all

    Name of the TensorFlow model folder, specified as a character vector or string scalar. modelFolder must be in the current folder, or you must include a full or relative path to the folder. modelFolder must contain the saved_model file with the .pb extension and the variables subfolder. The folder name can also contain the assets and assets.extra subfolders.

    • The saved_model file contains the layer graph architecture and training options (for example, optimizer, losses, and metrics).

    • The variables subfolder contains the weights learned by the pretrained TensorFlow network. By default, importNetworkFromTensorFlow imports the weights.

    • The assets subfolder contains supplementary files (for example, vocabularies), which the layer graph can use. importNetworkFromTensorFlow does not import the files in assets.

    • The assets.extra subfolder contains supplementary files, which coexist with the layer graph.

    For more information about saving a TensorFlow in SavedModel format, see Save TensorFlow Model.

    Example: "MobileNet"

    Example: "./MobileNet"

    Name-Value Arguments

    collapse all

    Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

    Example: importNetworkFromTensorflow(modelFolder,PreferredNestingType="customlayer") imports the network in modelFolder and uses custom layers to represent network composition.

    Name of the custom layers namespace in which importNetworkFromTensorFlow saves custom layers, specified as a character vector or string scalar. importNetworkFromTensorFlow saves the custom layers namespace in the current folder. If you do not specify Namespace, then importNetworkFromTensorFlow saves the custom layers in a namespace named +modelFolder in the current folder. For more information about namespaces, see Create Namespaces.

    importNetworkFromTensorFlow tries to generate a custom layer when you import a custom TensorFlow layer or when the software cannot convert a TensorFlow layer into an equivalent built-in MATLAB® layer. importNetworkFromTensorFlow saves each generated custom layer to a separate MATLAB code file in +Namespace. To view or edit a custom layer, open the associated MATLAB code file. For more information about custom layers, see Custom Layers.

    The +Namespace namespace can also contain the +ops inner namespace. This inner namespace contains MATLAB functions corresponding to TensorFlow operators that the automatically generated custom layers use. importNetworkFromTensorFlow saves the associated MATLAB function for each operator in a separate MATLAB code file in the +ops inner namespace. The object functions of dlnetwork, such as the predict function, use these operators when it interacts with the custom layers.

    Example: Namespace="MobileNet"

    Example: Namespace="CustomLayers"

    Network composition representation, specified as one of the following values:

    • "networklayer" — Represents network composition in the imported network using networkLayer layer objects.

    • "customlayer" — Represents network composition in the imported network using nested custom layers. For more information about custom layers, see Define Custom Deep Learning Layers.

    Example: PreferredNestingType="customlayer"

    Data Types: char | string

    Output Arguments

    collapse all

    Pretrained TensorFlow network, returned as an initialized dlnetwork object. In some cases, the software indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

    Limitations

    • importNetworkFromTensorFlow has been tested for TensorFlow versions 2.0 to 2.15.

    • importNetworkFromTensorFlow supports models created with the Keras 2 API. New features introduced in the Keras 3 API are unsupported.

    More About

    collapse all

    Tips

    • To use a pretrained network for prediction or transfer learning on new images, you must preprocess your images in the same way as the images that you use to train the imported model. The most common preprocessing steps are resizing images, subtracting image average values, and converting the images from BGR format to RGB format.

      • To resize images, use imresize. For example, imresize(image,[227 227 3]).

      • To convert images from RGB to BGR format, use flip. For example, flip(image,3).

      For more information about preprocessing images for training and prediction, see Preprocess Images for Deep Learning.

    • The members of the +Namespace namespace (custom layers and TensorFlow operators) are not accessible if the namespace parent folder is not on the MATLAB path. For more information, see Namespaces and the MATLAB Path.

    • MATLAB uses one-based indexing, whereas Python® uses zero-based indexing. In other words, the first element in an array has an index of 1 and 0 in MATLAB and Python, respectively. For more information about MATLAB indexing, see Array Indexing. In MATLAB, to use an array of indices (ind) created in Python, convert the array to ind+1.

    • For more tips, see Tips on Importing Models from TensorFlow, PyTorch, and ONNX.

    Algorithms

    The importNetworkFromTensorFlow function imports a TensorFlow layer into MATLAB by trying these steps in order:

    1. The function tries to import the TensorFlow layer as a built-in MATLAB layer. For more information, see TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers.

    2. The function tries to import the TensorFlow layer as a built-in MATLAB function. For more information, see Supported TensorFlow Operators.

    3. The function tries to import the TensorFlow layer as a custom layer. importNetworkFromTensorFlow saves the generated custom layers and the associated functions in the +Namespace namespace. For an example, see Import TensorFlow Network with Automatically Generated Custom Layers.

    4. The function imports the TensorFlow layer as a custom layer with a placeholder function. importNetworkFromTensorFlow saves the placeholder function in the +ops inner namespace of the +Namespace namespace. You must complete the placeholder function before you can use the network.

    In some cases, the software will indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

    Alternative Functionality

    App

    You can also import networks from external platforms by using the Deep Network Designer app. The app uses the importNetworkFromTensorFlow function to import the network, and displays a progress dialog box. During the import process, the app adds an input layer to the network, if possible, and displays an import report with details about any issues that require attention. After importing a network, you can interactively edit, visualize, and analyze the network. When you are finished editing the network, you can export it to Simulink® or generate MATLAB code for building networks.

    Block

    You can also work with TensorFlow networks using the TensorFlow Model Predict block. The TensorFlow Model Predict block additionally allows you to load TensorFlow functions to preprocess and postprocess data, and to configure input and output ports interactively.

    References

    [1] TensorFlow. “TensorFlow.” Accessed July 3, 2023. https://www.tensorflow.org/.

    [2] TensorFlow. “Using the SavedModel Format | TensorFlow Core.” Accessed July 3, 2023. https://www.tensorflow.org/guide/saved_model.

    Version History

    Introduced in R2023b

    expand all

    See Also

    | | | | | |

    Topics