importNetworkFromTensorFlow - Import TensorFlow network as MATLAB network - MATLAB (original) (raw)

Import TensorFlow network as MATLAB network

Since R2023b

Syntax

Description

[net](#mw%5Fa4f42437-381a-4324-bbd9-121fc4c9d65f) = importNetworkFromTensorFlow([modelFolder](#mw%5F3278672e-9100-4b2e-aa31-3a4523f9f08e)) imports a pretrained TensorFlow™ network from the folder modelFolder, which contains the network in the SavedModel format. The network comprises the layers defined in thesaved_model file with the .pb extension and the learned weights in the variables subfolder. The function returns the network net as an initialized dlnetwork object.

importNetworkFromTensorFlow requires the Deep Learning Toolbox™ Converter for TensorFlow Models support package. If this support package is not installed, thenimportNetworkFromTensorFlow provides a download link.

Note

The importNetworkFromTensorFlow function can generate a custom layer when importing a TensorFlow layer. For more information, see Algorithms. The function saves the generated custom layers in the + modelFolder namespace.

example

[net](#mw%5Fa4f42437-381a-4324-bbd9-121fc4c9d65f) = importNetworkFromTensorFlow([modelFolder](#mw%5F3278672e-9100-4b2e-aa31-3a4523f9f08e),[Name=Value](#namevaluepairarguments)) specifies additional options using one or more name-value arguments. For example, you can specify a namespace in which to save the generated custom layers and associated functions.

example

Examples

collapse all

Import TensorFlow Network and Classify Image

Import a pretrained TensorFlow network in the SavedModel format and use the imported network to predict class labels.

Specify the model folder.

if ~exist('digitsdlnetwork','dir') unzip('digitsdlnetwork.zip') end modelFolder = './digitsdlnetwork';

Specify the class names.

classNames = {'0','1','2','3','4','5','6','7','8','9'};

Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model... Translating the model, this may take a few minutes... Finished translation. Assembling network... Import finished.

net = dlnetwork with properties:

     Layers: [12x1 nnet.cnn.layer.Layer]
Connections: [12x2 table]
 Learnables: [6x3 table]
      State: [0x3 table]
 InputNames: {'input_1'}
OutputNames: {'activation_1'}
Initialized: 1

View summary with summary.

Read the image you want to classify and display the size of the image. The image is a grayscale (one-channel) image with a size of 28-by-28 pixels.

digitDatasetPath = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset'); I = imread(fullfile(digitDatasetPath,'5','image4009.png')); size(I)

Display the input size of the network. In this case, the image size matches the network input size. If the sizes do not match, you must resize the image using this command: imresize(I,netInputSize(1:2)).

netInputSize = net.Layers(1).InputSize

netInputSize = 1×3

28    28     1

Convert the image to a dlarray object. Format the images with the dimensions 'SSCB' (spatial, spatial, channel, batch). In this case, the batch size is 1 and you can omit it ('SSC').

I_dlarray = dlarray(single(I),'SSC');

Classify the sample image and find the predicted label.

prob = predict(net,I_dlarray); [~,label] = max(prob);

Display the image and the classification result.

imshow(I) title(['Classification result ' classNames{label}])

Import TensorFlow Network with Automatically Generated Custom Layers

Import a pretrained TensorFlow network in the SavedModel format. The network contains layers that the software cannot convert to built-in MATLAB layers. The software automatically generates custom layers when you import this network.

This example uses the findCustomLayers helper function. To view the code for this function, see Helper Function.

Specify the model folder.

if ~exist('digitsdlnetworkwithnoise','dir') unzip('digitsdlnetworkwithnoise.zip') end modelFolder = './digitsdlnetworkwithnoise';

Import a TensorFlow network in the SavedModel format.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model... Translating the model, this may take a few minutes... Finished translation. Assembling network... Import finished.

net = dlnetwork with properties:

     Layers: [14×1 nnet.cnn.layer.Layer]
Connections: [14×2 table]
 Learnables: [6×3 table]
      State: [0×3 table]
 InputNames: {'input_1'}
OutputNames: {'activation_1'}
Initialized: 1

View summary with summary.

The imported network contains layers the software cannot convert to built-in MATLAB layers, so importNetworkFromTensorFlow automatically generates custom layers in place of these layers. The function saves each generated custom layer to a separate M file in the +digitsdlnetworkwithnoise namespace in the current folder.

Find the indices of the automatically generated custom layers using the helper function findCustomLayers, and display the custom layers.

ind = findCustomLayers(net.Layers,'+digitsdlnetworkwithnoise'); net.Layers(ind)

ans = 3×1 Layer array with layers:

 1   'concatenate_1'      Concatenate     digitsdlnetworkwithnoise.kConcatenate1Layer3826
 2   'gaussian_noise_1'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise1Layer3766
 3   'gaussian_noise_2'   GaussianNoise   digitsdlnetworkwithnoise.kGaussianNoise2Layer3791

Plot the network architecture.

plot(net) title('Network Architecture')

Helper Function

This section defines the findCustomLayers helper function. findCustomLayers returns the indices of the custom layers that importNetworkFromTensorFlow automatically generates.

function indices = findCustomLayers(layers,Namespace)

s = what(['.' filesep Namespace]);

indices = zeros(1,length(s.m)); for i = 1:length(layers) for j = 1:length(s.m) if strcmpi(class(layers(i)),[Namespace(2:end) '.' s.m{j}(1:end-2)]) indices(j) = i; end end end

end

Import and Initialize TensorFlow Network

Import a pretrained TensorFlow network in the SavedModel format, which results in an uninitialized network. Then, initialize the network.

Specify the model folder.

if ~exist('imageClassificationNet','dir') unzip('imageClassificationNet.zip') end modelFolder = './imageClassificationNet';

Import the model using the importNetworkFromTensorFlow function. Because this model does not have a fixed input size, the function imports the model as an uninitialized dlnetwork object without an input layer. The software displays a warning that describes how to initialize the network.

net = importNetworkFromTensorFlow(modelFolder)

Importing the saved model... Translating the model, this may take a few minutes...

Warning: Size of the input to "input_1" is unknown in TensorFlow.

Finished translation. Assembling network...

Warning: Network is imported as an uninitialized dlnetwork object. Before using the network, add input layer(s):

inputLayer1 = imageInputLayer(, Normalization="none"); net = addInputLayer(net, inputLayer1, Initialize=true);

For more information on which format(s) to use, see documentation for dlarray.

net = dlnetwork with properties:

     Layers: [7x1 nnet.cnn.layer.Layer]
Connections: [6x2 table]
 Learnables: [6x3 table]
      State: [2x3 table]
 InputNames: {'conv2d'}
OutputNames: {'softmax'}
Initialized: 0

View summary with summary.

Specify the input size of the imported network and create an image input layer. Then, add the image input layer to the imported network and initialize the network using the addInputLayer function.

inputLayer1 = imageInputLayer([28 28 3], Normalization="none"); net = addInputLayer(net,inputLayer1,Initialize=true)

net = dlnetwork with properties:

     Layers: [8x1 nnet.cnn.layer.Layer]
Connections: [7x2 table]
 Learnables: [6x3 table]
      State: [2x3 table]
 InputNames: {'imageinput'}
OutputNames: {'softmax'}
Initialized: 1

View summary with summary.

The network contains the input layer and is ready to use for prediction. You can analyze the network further using the Deep Network Designer app.

Import Network with Nested Networks

Import a TensorFlow network in the SavedModel format as an initialized dlnetwork object. Analyze the network using the analyzeNetwork function.

net = importNetworkFromTensorFlow("CustomConvReluBN");

Importing the saved model... Translating the model, this may take a few minutes... Finished translation. Assembling network... Import finished.

net is a dlnetwork object with seven layers. The second and third layers are networkLayer objects that each contain a nested neural network.

Expand the networkLayer layer using the expandLayers function and analyze the network.

netExpanded = expandLayers(net); analyzeNetwork(netExpanded)

The app shows that the expanded network netExpanded has four more layers than net.

Import the TensorFlow network again, using custom layers to represent the nested neural network. Analyze the network.

netCustom = importNetworkFromTensorFlow("CustomConvReluBN",PreferredNestingType="customlayer");

Importing the saved model... Translating the model, this may take a few minutes...

Warning: The namespace "CustomConvReluBN" already exists in the Current Folder, and will be overwritten.

Finished translation. Assembling network... Import finished.

analyzeNetwork(netCustom)

netCustom contains seven layers. The second and third layers are custom layers that represent nested neural networks.

Input Arguments

collapse all

modelFolder — Name of TensorFlow model folder

character vector | string scalar

Name of the TensorFlow model folder, specified as a character vector or string scalar.modelFolder must be in the current folder, or you must include a full or relative path to the folder. modelFolder must contain thesaved_model file with the .pb extension and thevariables subfolder. The folder name can also contain theassets and assets.extra subfolders.

• The saved_model file contains the layer graph architecture and training options (for example, optimizer, losses, and metrics).
• The variables subfolder contains the weights learned by the pretrained TensorFlow network. By default, importNetworkFromTensorFlow imports the weights.
• The assets subfolder contains supplementary files (for example, vocabularies), which the layer graph can use.importNetworkFromTensorFlow does not import the files inassets.
• The assets.extra subfolder contains supplementary files, which coexist with the layer graph.

Example: "MobileNet"

Example: "./MobileNet"

Name-Value Arguments

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: importNetworkFromTensorflow(modelFolder,PreferredNestingType="customlayer") imports the network in modelFolder and uses custom layers to represent network composition.

Namespace — Name of custom layers namespace

character vector | string scalar

Name of the custom layers namespace in which importNetworkFromTensorFlow saves custom layers, specified as a character vector or string scalar.importNetworkFromTensorFlow saves the custom layers namespace in the current folder. If you do not specify Namespace, thenimportNetworkFromTensorFlow saves the custom layers in a namespace named+[modelFolder](#mw%5F3278672e-9100-4b2e-aa31-3a4523f9f08e) in the current folder. For more information about namespaces, see Create Namespaces.

importNetworkFromTensorFlow tries to generate a custom layer when you import a custom TensorFlow layer or when the software cannot convert a TensorFlow layer into an equivalent built-in MATLAB® layer. importNetworkFromTensorFlow saves each generated custom layer to a separate MATLAB code file in +`Namespace`. To view or edit a custom layer, open the associated MATLAB code file. For more information about custom layers, see Custom Layers.

The +`Namespace` namespace can also contain the +ops inner namespace. This inner namespace contains MATLAB functions corresponding to TensorFlow operators that the automatically generated custom layers use.importNetworkFromTensorFlow saves the associated MATLAB function for each operator in a separate MATLAB code file in the +ops inner namespace. The object functions of dlnetwork, such as the predict function, use these operators when it interacts with the custom layers.

Example: Namespace="MobileNet"

Example: Namespace="CustomLayers"

PreferredNestingType — Network composition representation

"networklayer" (default) | "customlayer"

Network composition representation, specified as one of the following values:

• "networklayer" — Represents network composition in the imported network using networkLayer layer objects.
• "customlayer" — Represents network composition in the imported network using nested custom layers. For more information about custom layers, see Define Custom Deep Learning Layers.

Example: PreferredNestingType="customlayer"

Data Types: char | string

Output Arguments

collapse all

net — Pretrained TensorFlow network

dlnetwork object

Pretrained TensorFlow network, returned as an initialized dlnetwork object. In some cases, the software indicates that you need to initialize the imported network. For an example, seeImport and Initialize TensorFlow Network.

More About

collapse all

TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers

importNetworkFromTensorFlow supports these TensorFlow-Keras layer types for conversion into built-in MATLAB layers, with some limitations.

Supported TensorFlow Operators

importNetworkFromTensorFlow supports these TensorFlow operators for conversion into MATLAB functions with dlarray support.

For more information about functions that operate on dlarray objects, see List of Functions with dlarray Support.

Generate Code for Imported Network

You can use MATLAB Coder™ or GPU Coder™ software with Deep Learning Toolbox software to generate MEX, standalone CPU, CUDA® MEX, or standalone CUDA code for an imported network. For more information, see Code Generation.

• Use MATLAB Coder with Deep Learning Toolbox to generate MEX or standalone CPU code that runs on desktop or embedded targets. You can deploy generated standalone code that uses the Intel® Math Kernel Library for Deep Neural Networks (MKL-DNN) or the ARM® Compute library. Alternatively, you can generate generic C or C++ code that does not call third-party library functions. For more information, see Deep Learning with MATLAB Coder (MATLAB Coder).
• Use GPU Coder with Deep Learning Toolbox to generate CUDA MEX or standalone CUDA code that runs on desktop or embedded targets. You can deploy generated standalone CUDA code that uses the CUDA deep neural network library (cuDNN), the TensorRT™ high performance inference library, or the ARM Compute library for Mali GPU. For more information, see Deep Learning with GPU Coder (GPU Coder).

importNetworkFromTensorFlow returns the networknet as a dlnetwork object. This object supports code generation. For more information about MATLAB Coder and GPU Coder support for Deep Learning Toolbox objects, see Supported Classes (MATLAB Coder) and Supported Classes (GPU Coder), respectively.

You can generate code for any imported network whose layers support code generation. For lists of the layers that support code generation with MATLAB Coder and GPU Coder, see Supported Layers (MATLAB Coder) and Supported Layers (GPU Coder), respectively. For more information about the code generation capabilities and limitations of each built-in MATLAB layer, see the Extended Capabilities section of the layer. For example, seeCode Generation and GPU Code Generation of imageInputLayer.

Use Imported Network on GPU

importNetworkFromTensorFlow does not execute on a GPU. However,importNetworkFromTensorFlow imports a pretrained neural network for deep learning as a dlnetwork object, which you can use on a GPU.

• You can make predictions with the imported network on a CPU or GPU by usingpredict. Thepredict function executes on the GPU if you store the input data or network parameters on the GPU.
  • If you use minibatchqueue to process and manage the mini-batches of input data, the minibatchqueue object converts the output to a GPU array by default if a GPU is available.
  • Use dlupdate to convert the learnable parameters of adlnetwork object to GPU arrays.
    net = dlupdate(@gpuArray,net)
• You can train the imported network on a CPU or GPU by using trainnet. To specify training options, including options for the execution environment, use thetrainingOptions function. Specify the hardware requirements using the ExecutionEnvironment name-value argument. For more information about how to accelerate training, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud.

Using a GPU requires a Parallel Computing Toolbox™ license and a supported GPU device. For information about supported devices, see GPU Computing Requirements (Parallel Computing Toolbox).

Tips

• To use a pretrained network for prediction or transfer learning on new images, you must preprocess your images in the same way as the images that you use to train the imported model. The most common preprocessing steps are resizing images, subtracting image average values, and converting the images from BGR format to RGB format.
  • To resize images, use imresize. For example, imresize(image,[227 227 3]).
  • To convert images from RGB to BGR format, use flip. For example,flip(image,3).
    For more information about preprocessing images for training and prediction, see Preprocess Images for Deep Learning.
• The members of the +[Namespace](#mw%5F1a70a028-9aea-42ff-b559-6d04b48d46cd) namespace (custom layers and TensorFlow operators) are not accessible if the namespace parent folder is not on the MATLAB path. For more information, see Namespaces and the MATLAB Path.
• MATLAB uses one-based indexing, whereas Python® uses zero-based indexing. In other words, the first element in an array has an index of 1 and 0 in MATLAB and Python, respectively. For more information about MATLAB indexing, see Array Indexing. In MATLAB, to use an array of indices (ind) created in Python, convert the array to ind+1.
• For more tips, see Tips on Importing Models from TensorFlow, PyTorch, and ONNX.

Algorithms

The importNetworkFromTensorFlow function imports a TensorFlow layer into MATLAB by trying these steps in order:

1. The function tries to import the TensorFlow layer as a built-in MATLAB layer. For more information, see TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers.
2. The function tries to import the TensorFlow layer as a built-in MATLAB function. For more information, see Supported TensorFlow Operators.
3. The function tries to import the TensorFlow layer as a custom layer. importNetworkFromTensorFlow saves the generated custom layers and the associated functions in the+[Namespace](#mw%5F1a70a028-9aea-42ff-b559-6d04b48d46cd) namespace. For an example, seeImport TensorFlow Network with Automatically Generated Custom Layers.
4. The function imports the TensorFlow layer as a custom layer with a placeholder function.importNetworkFromTensorFlow saves the placeholder function in the+ops inner namespace of the+`Namespace` namespace. You must complete the placeholder function before you can use the network.

In some cases, the software will indicates that you need to initialize the imported network. For an example, see Import and Initialize TensorFlow Network.

Alternative Functionality

App

You can also import models from external platforms using the Deep Network Designer app. The app uses the importNetworkFromTensorFlow function to import the network. On import, the app shows an import report with details about any issues that require attention.

References

Version History

Introduced in R2023b

expand all

R2024b: Represent network composition using networkLayer

You can import a network that uses networkLayer objects to represent network composition. To specify whether the imported network represents composition using networkLayer or custom layer objects, use thePreferredNestingType name-value argument. For more information, seeDeep Learning Network Composition.

R2024b: Import networks with new layers and operators

You can import the following TensorFlow layers into Deep Learning Toolbox layers:

• tf.keras.layers.Identity
• tf.keras.layers.LayerNormalization
• tf.keras.layers.SpatialDropout1D
• tf.keras.layers.SpatialDropout2D
• tf.keras.layers.SpatialDropout3D

You can also import the following operators into custom layers:

• tf.raw_ops.Erf
• tf.raw_ops.ResourceGather

R2024a: Import Multi-Input-Multi-Output (MIMO) TFOpLambda layers

You can now import a TensorFlow network that includes a TensorFlow-Keras TFOpLambda layer with multiple inputs and outputs.

R2024a: Import networks that include TimeDistributed layers

importNetworkFromTensorFlow now supports importing networks that includetf.keras.layers.TimeDistributed layers.

R2024a: Improved network performance for custom layers

importNetworkFromTensorFlow generates networks with improved inference and training performance for custom layers.