initialize - Initialize learnable and state parameters of neural network - MATLAB (original) (raw)

Initialize learnable and state parameters of neural network

Since R2021a

Syntax

Description

Tip

Most dlnetwork objects are initialized by default. You only need to manually initialize a dlnetwork if it is uninitialized. You can check if a network is initialized using the Initialized property of thedlnetwork object.

[netUpdated](#mw%5Fe20890bc-65a0-4676-bc78-ca8b72740f72) = initialize([net](#mw%5F9a743689-7093-4df8-a6e8-44399d76dbc5)) initializes any unset learnable parameters and state values of net based on the input sizes defined by the network input layers. Any learnable or state parameters that already contain values remain unchanged.

A network with unset, empty values for learnable and state parameters is_uninitialized_. You must initialize an uninitializeddlnetwork before you can use it. By default, dlnetwork objects are constructed with initial parameters and do not need initializing.

example

[netUpdated](#mw%5Fe20890bc-65a0-4676-bc78-ca8b72740f72) = initialize([net](#mw%5F9a743689-7093-4df8-a6e8-44399d76dbc5),[X1,...,XN](#mw%5F3af1d784-472c-4f9b-872d-d8c26929d3e7)) initializes any unset learnable parameters and state values of net based on the example network inputs or network data layout objects X1,...,XN. Use this syntax when the network has inputs that are not connected to an input layer.

example

Examples

collapse all

Define a simple image classification network as a layer array.

layers = [ imageInputLayer([28 28 1],Normalization="none") convolution2dLayer(5,20) batchNormalizationLayer reluLayer fullyConnectedLayer(10) softmaxLayer];

Convert the layer graph to a dlnetwork object. Create an uninitialized dlnetwork object by setting the Initialize option to false.

net = dlnetwork(layers,Initialize=false);

View the learnable parameters of the network. Because the network is not initialized, the values are empty.

ans=6×3 table Layer Parameter Value
___________ _________ ____________

"conv"         "Weights"    {0×0 double}
"conv"         "Bias"       {0×0 double}
"batchnorm"    "Offset"     {0×0 double}
"batchnorm"    "Scale"      {0×0 double}
"fc"           "Weights"    {0×0 double}
"fc"           "Bias"       {0×0 double}

Initialize the learnable parameters of the network using the initialize function.

View the learnable parameters of the network. Because the network is now initialized, the values are nonempty with sizes inferred using the size of the input layer.

ans=6×3 table Layer Parameter Value
___________ _________ ___________________

"conv"         "Weights"    { 5×5×1×20 dlarray}
"conv"         "Bias"       { 1×1×20   dlarray}
"batchnorm"    "Offset"     { 1×1×20   dlarray}
"batchnorm"    "Scale"      { 1×1×20   dlarray}
"fc"           "Weights"    {10×11520  dlarray}
"fc"           "Bias"       {10×1      dlarray}

Define a multi-input image classification network.

numFilters = 24;

net = dlnetwork;

layersBranch1 = [ convolution2dLayer(3,6*numFilters,Padding="same",Stride=2) groupNormalizationLayer("all-channels") reluLayer convolution2dLayer(3,numFilters,Padding="same") groupNormalizationLayer("channel-wise") additionLayer(2,Name="add") reluLayer fullyConnectedLayer(10) softmaxLayer];

layersBranch2 = [ convolution2dLayer(1,numFilters,Name="conv_branch") groupNormalizationLayer("all-channels",Name="groupnorm_branch")];

net = addLayers(net, layersBranch1); net = addLayers(net,layersBranch2); net = connectLayers(net,"groupnorm_branch","add/in2");

Visualize the layers in a plot.

Figure contains an axes object. The axes object contains an object of type graphplot.

View the learnable parameters of the network. Because the network is not initialized, the values are empty.

ans=14×3 table Layer Parameter Value
__________________ _________ ____________

"conv_1"              "Weights"    {0×0 double}
"conv_1"              "Bias"       {0×0 double}
"groupnorm_1"         "Offset"     {0×0 double}
"groupnorm_1"         "Scale"      {0×0 double}
"conv_2"              "Weights"    {0×0 double}
"conv_2"              "Bias"       {0×0 double}
"groupnorm_2"         "Offset"     {0×0 double}
"groupnorm_2"         "Scale"      {0×0 double}
"fc"                  "Weights"    {0×0 double}
"fc"                  "Bias"       {0×0 double}
"conv_branch"         "Weights"    {0×0 double}
"conv_branch"         "Bias"       {0×0 double}
"groupnorm_branch"    "Offset"     {0×0 double}
"groupnorm_branch"    "Scale"      {0×0 double}

View the names of the network inputs.

ans = 1×2 cell {'conv_1'} {'conv_branch'}

Create random dlarray objects representing inputs to the network. Use an example input of size 64-by-64 with 3 channels for the main branch of the network. Use an input of size 64-by-64 with 18 channels for the second branch.

inputSize = [64 64 3]; inputSizeBranch = [32 32 18];

X1 = dlarray(rand(inputSize),"SSCB"); X2 = dlarray(rand(inputSizeBranch),"SSCB");

Initialize the learnable parameters of the network using the initialize function and specify the example inputs. Specify the inputs with order corresponding to the InputNames property of the network.

net = initialize(net,X1,X2);

View the learnable parameters of the network. Because the network is now initialized, the values are nonempty with sizes inferred using the size of the input data.

ans=14×3 table Layer Parameter Value
__________________ _________ _____________________

"conv_1"              "Weights"    { 3×3×3×144  dlarray}
"conv_1"              "Bias"       { 1×1×144    dlarray}
"groupnorm_1"         "Offset"     { 1×1×144    dlarray}
"groupnorm_1"         "Scale"      { 1×1×144    dlarray}
"conv_2"              "Weights"    { 3×3×144×24 dlarray}
"conv_2"              "Bias"       { 1×1×24     dlarray}
"groupnorm_2"         "Offset"     { 1×1×24     dlarray}
"groupnorm_2"         "Scale"      { 1×1×24     dlarray}
"conv_branch"         "Weights"    { 1×1×18×24  dlarray}
"conv_branch"         "Bias"       { 1×1×24     dlarray}
"groupnorm_branch"    "Offset"     { 1×1×24     dlarray}
"groupnorm_branch"    "Scale"      { 1×1×24     dlarray}
"fc"                  "Weights"    {10×24576    dlarray}
"fc"                  "Bias"       {10×1        dlarray}

Create an uninitialized dlnetwork object that has two unconnected inputs.

layers = [ convolution2dLayer(5,16,Name="conv") batchNormalizationLayer reluLayer fullyConnectedLayer(50) flattenLayer concatenationLayer(1,2,Name="cat") fullyConnectedLayer(10) softmaxLayer];

net = dlnetwork(layers,Initialize=false);

View the input names of the network.

ans = 1×2 cell {'conv'} {'cat/in2'}

Create network data layout objects that represent input data for the inputs. For the first input, specify a batch of 28-by-28 grayscale images. For the second input specify a batch of single-channel feature data.

layout1 = networkDataLayout([28 28 1 NaN],"SSCB"); layout2 = networkDataLayout([1 NaN],"CB");

Initialize the network using the network data layout objects.

net = initialize(net,layout1,layout2)

net = dlnetwork with properties:

     Layers: [8×1 nnet.cnn.layer.Layer]
Connections: [7×2 table]
 Learnables: [8×3 table]
      State: [2×3 table]
 InputNames: {'conv'  'cat/in2'}
OutputNames: {'softmax'}
Initialized: 1

View summary with summary.

Input Arguments

collapse all

Uninitialized network, specified as a dlnetwork object.

Example network inputs or data layouts to use to determine the size and formats of learnable and state parameters, each specified as one of these values:

Formatted dlarray object
Formatted networkDataLayout object
Unformatted dlarray object (since R2025a)
Unformatted networkDataLayout object (since R2025a)

The software propagates X1,...XN through the network to determine the appropriate sizes and formats of the learnable and state parameters of thedlnetwork object and initializes any unset learnable or state parameters.

To create a neural network that receives unformatted data, use an inputLayer object and do not specify a format. (since R2025a)

Before R2025a: X1,...XN must be formatteddlarray or networkDataLayout objects.

Provide example inputs in the same order as the order specified by theInputNames property of the input network.

Note

Automatic initialization uses only the size and format information of the input data. For initialization that depends on the values on the input data, you must initialize the learnable parameters manually.

Output Arguments

collapse all

Initialized network, returned as an initialized dlnetwork object.

The initialize function does not preserve quantization information. If the input network is a quantized network, then the output network does not contain quantization information.

Version History

Introduced in R2021a

expand all

Initialize neural networks with unformatted data by specifying the inputs X1,...,XN as unformatted dlarray or unformatted networkDataLayout objects.

Input layers such as imageInputLayer and sequenceInputLayer contain properties that networks use for data normalization. These properties are Mean,StandardDeviation, Min, andMax. The software uses these properties to apply the data normalization method defined by the Normalization property of the layer.

Starting in R2023b, when you initialize a network by creating an initialized dlnetwork or by using the initialize function, the software initializes the Mean,StandardDeviation, Min, andMax properties of input layers if you do not set them when you create the layer and if the normalization method requires them. For normalization methods that use two properties, for example, zscore, the software initializes those properties only if you do not set either property when you create the layer.

For zerocenter normalization, Mean is initialized to 0.
For zscore normalization, Mean is initialized to 0 and StandardDeviation is initialized to 1.
For rescale-symmetric normalization, Min is initialized to -1 and Max is initialized to 1.
For rescale-zero-one normalization, Min is initialized to 0 and Max is initialized to1.

By default, the software automatically calculates the normalization statistics during training. To customize the normalization, set the Mean,StandardDeviation, Min, andMax properties of input layers manually.

In previous releases, the software errors when you initialize a network containing an input layer that uses a normalization method requiring properties that you do not specify when you create the layer.