getLearnRateFactor - Get learn rate factor of layer learnable parameter - MATLAB (original) (raw)

Get learn rate factor of layer learnable parameter

Syntax

Description

[factor](#mw%5F30cab7b2-7138-4713-8091-3df5fb403b4a) = getLearnRateFactor([layer](#d126e106057),[parameterName](#d126e106071)) returns the learn rate factor of the learnable parameter with the nameparameterName in layer.

For built-in layers, you can get the learn rate factor directly by using the corresponding property. For example, for a convolution2dLayer layer, the syntax factor = getLearnRateFactor(layer,'Weights') is equivalent tofactor = layer.WeightLearnRateFactor.

example

[factor](#mw%5F30cab7b2-7138-4713-8091-3df5fb403b4a) = getLearnRateFactor([layer](#d126e106057),[parameterPath](#mw%5F3851b022-39bc-4f15-9475-1bdaf1fe6849%5Fsep%5Fmw%5Fbb76b081-6fd1-4a89-9ec7-fd4394ad67fb)) returns the learn rate factor of the parameter specified by the pathparameterPath. Use this syntax when the layer is anetworkLayer or when the parameter is in adlnetwork object in a custom layer.

example

[factor](#mw%5F30cab7b2-7138-4713-8091-3df5fb403b4a) = getLearnRateFactor([net](#mw%5F3851b022-39bc-4f15-9475-1bdaf1fe6849%5Fsep%5Fmw%5F3ffb42b3-6af7-4c22-82b3-4fb0f6a399f2),[layerName](#mw%5F3851b022-39bc-4f15-9475-1bdaf1fe6849%5Fsep%5Fmw%5F5e8103c3-6a84-4c6d-a782-b16533603493),[parameterName](#d126e106071)) returns the learn rate factor of the parameter with the nameparameterName in the layer with namelayerName for the specified dlnetwork object.

example

[factor](#mw%5F30cab7b2-7138-4713-8091-3df5fb403b4a) = getLearnRateFactor([net](#mw%5F3851b022-39bc-4f15-9475-1bdaf1fe6849%5Fsep%5Fmw%5F3ffb42b3-6af7-4c22-82b3-4fb0f6a399f2),[parameterPath](#mw%5F3851b022-39bc-4f15-9475-1bdaf1fe6849%5Fsep%5Fmw%5Fbb76b081-6fd1-4a89-9ec7-fd4394ad67fb)) returns the learn rate factor of the parameter specified by the pathparameterPath. Use this syntax when the parameter is in a networkLayer or when the parameter is in adlnetwork object in a custom layer..

example

Examples

collapse all

Set and get the learning rate factor of a learnable parameter of a custom SReLU layer.

Create a layer array containing the custom layer sreluLayer, attached to this example as a supporting file. To access this layer, open this example as a live script.

layers = [ ... imageInputLayer([28 28 1]) convolution2dLayer(5,20) batchNormalizationLayer sreluLayer fullyConnectedLayer(10) softmaxLayer];

Set the learn rate factor of the LeftThreshold learnable parameter of the sreluLayer to 2.

layers(4) = setLearnRateFactor(layers(4),"LeftThreshold",2);

View the updated learn rate factor.

factor = getLearnRateFactor(layers(4),"LeftThreshold")

Set and get the learning rate factor of a learnable parameter of a nested layer defined using network composition.

Create a residual block layer using the custom layer residualBlockLayer attached to this example as a supporting file. To access this file, open this example as a Live Script.

numFilters = 64; layer = residualBlockLayer(numFilters)

layer = residualBlockLayer with properties:

   Name: ''

Learnable Parameters Network: [1×1 dlnetwork]

State Parameters Network: [1×1 dlnetwork]

Show all properties

View the layers of the nested network.

ans = 7×1 Layer array with layers:

 1   'conv_1'        2-D Convolution       64 3×3 convolutions with stride [1  1] and padding 'same'
 2   'batchnorm_1'   Batch Normalization   Batch normalization
 3   'relu_1'        ReLU                  ReLU
 4   'conv_2'        2-D Convolution       64 3×3 convolutions with stride [1  1] and padding 'same'
 5   'batchnorm_2'   Batch Normalization   Batch normalization
 6   'add'           Addition              Element-wise addition of 2 inputs
 7   'relu_2'        ReLU                  ReLU

Set the learning rate factor of the learnable parameter 'Weights' of the layer 'conv_1' to 2 using the setLearnRateFactor function.

factor = 2; layer = setLearnRateFactor(layer,'Network/conv_1/Weights',factor);

Get the updated learning rate factor using the getLearnRateFactor function.

factor = getLearnRateFactor(layer,'Network/conv_1/Weights')

Set and get the learning rate factor of a learnable parameter of a dlnetwork object.

Create a dlnetwork object

net = dlnetwork;

layers = [ imageInputLayer([28 28 1],Normalization="none",Name="in") convolution2dLayer(5,20,Name="conv") batchNormalizationLayer(Name="bn") reluLayer(Name="relu") fullyConnectedLayer(10,Name="fc") softmaxLayer(Name="sm")];

net = addLayers(net,layers);

Set the learn rate factor of the 'Weights' learnable parameter of the convolution layer to 2 using the setLearnRateFactor function.

factor = 2; net = setLearnRateFactor(net,'conv',Weights=factor);

Get the updated learn rate factor using the getLearnRateFactor function.

factor = getLearnRateFactor(net,'conv',"Weights")

Create an array of layers containing an lstmLayer with 100 hidden units and a dropoutLayer with a dropout probability of 0.2.

layers = [lstmLayer(100,OutputMode="sequence",Name="lstm") dropoutLayer(0.2,Name="dropout")];

Create a network layer containing these layers.

lstmDropoutLayer = networkLayer(layers,Name="lstmDropout");

Use the network layer to build a network.

layers = [sequenceInputLayer(3) lstmDropoutLayer lstmDropoutLayer fullyConnectedLayer(10) softmaxLayer];

Create a dlnetwork object. You can also create a dlnetwork object by training the network using the trainnet function.

Set the learning rate factor of the InputWeights learnable parameter of the LSTM layer in the first network layer to 2 using the setLearnRateFactor function.

factor = 2; net = setLearnRateFactor(net,"lstmDropout_1/lstm/InputWeights",factor);

Get the updated learning rate factor using the getLearnRateFactor function.

factor = getLearnRateFactor(net,"lstmDropout_1/lstm/InputWeights")

Set and get the learning rate factor of a learnable parameter of a custom nested layer defined using network composition in a dlnetwork object.

Create a dlnetwork object containing the custom layer residualBlockLayer attached to this example as a supporting file. To access this file, open this example as a Live Script.

inputSize = [224 224 3]; numFilters = 32; numClasses = 5;

layers = [ imageInputLayer(inputSize,'Normalization','none','Name','in') convolution2dLayer(7,numFilters,'Stride',2,'Padding','same','Name','conv') groupNormalizationLayer('all-channels','Name','gn') reluLayer('Name','relu') maxPooling2dLayer(3,'Stride',2,'Name','max') residualBlockLayer(numFilters,'Name','res1') residualBlockLayer(numFilters,'Name','res2') residualBlockLayer(2numFilters,'Stride',2,'IncludeSkipConvolution',true,'Name','res3') residualBlockLayer(2numFilters,'Name','res4') residualBlockLayer(4numFilters,'Stride',2,'IncludeSkipConvolution',true,'Name','res5') residualBlockLayer(4numFilters,'Name','res6') globalAveragePooling2dLayer('Name','gap') fullyConnectedLayer(numClasses,'Name','fc') softmaxLayer('Name','sm')];

dlnet = dlnetwork(layers);

View the layers of the nested network in the layer 'res1'.

dlnet.Layers(6).Network.Layers

ans = 7×1 Layer array with layers:

 1   'conv_1'        2-D Convolution       32 3×3×32 convolutions with stride [1  1] and padding 'same'
 2   'batchnorm_1'   Batch Normalization   Batch normalization with 32 channels
 3   'relu_1'        ReLU                  ReLU
 4   'conv_2'        2-D Convolution       32 3×3×32 convolutions with stride [1  1] and padding 'same'
 5   'batchnorm_2'   Batch Normalization   Batch normalization with 32 channels
 6   'add'           Addition              Element-wise addition of 2 inputs
 7   'relu_2'        ReLU                  ReLU

Set the learning rate factor of the learnable parameter 'Weights' of the layer 'conv_1' to 2 using the setLearnRateFactor function.

factor = 2; dlnet = setLearnRateFactor(dlnet,'res1/Network/conv_1/Weights',factor);

Get the updated learning rate factor using the getLearnRateFactor function.

factor = getLearnRateFactor(dlnet,'res1/Network/conv_1/Weights')

Input Arguments

collapse all

Input layer, specified as a scalar Layer object.

Parameter name, specified as a character vector or a string scalar.

Path to parameter in nested layer, specified as a string scalar or a character vector. A nested layer can be a layer within a networkLayer or a custom layer that itself defines a neural network as a learnable parameter.

If the input to getLearnRateFactor is a layer, then:

If the nested layer is in a network layer, the parameter path has the form"nestedLayerName/parameterName" wherenestedlayerName is the name of the nested layer inside the network layer, and parameterName is the name of the parameter. If there are multiple levels of nested layers, then specify the path using the formnestedLayerName1/.../nestedLayerNameN/parameterName.
If the nested layer is a custom layer that itself defines a neural network as a learnable parameter, the parameter path has the form"propertyName/layerName/parameterName" wherepropertyName is the name of the property containing adlnetwork object, layerName is the name of the layer in the dlnetwork object, andparameterName is the name of the parameter. If there are multiple levels of nested layers, then specify the path using the form"propertyName1/layerName1/.../propertyNameN/layerNameN/parameterName".

If the input to getLearnRateFactor is a dlnetwork object and the desired parameter is in a nested layer, then:

If the nested layer is in a network layer, the parameter path has the form"networkLayerName/nestedLayerName/parameterName" wherenetworkLayerName is the name of the network layer,nestedlayerName is the name of the nested layer inside the network layer, and parameterName is the name of the parameter. If there are multiple levels of nested layers, then specify the path using the form"networkLayerName1/.../networkLayerNameN/nestedLayerName/parameterName".
If the nested layer is a custom layer that itself defines a neural network as a learnable parameter, the parameter path has the form"customLayerName1/propertyName/layerName/parameterName", where layerName1 is the name of the layer in the inputdlnetwork object, propertyName is the name of the property of the layer containing a dlnetwork object, layerName is the name of the layer in thedlnetwork object, and parameterName is the name of the parameter. If there are multiple levels of nested layers, then specify the path using the form"customLayerName1/propertyName1/.../customLayerNameN/propertyNameN/layerName/parameterName".

Data Types: char | string

Neural network, specified as a dlnetwork object.

Layer name, specified as a string scalar or a character vector.

Data Types: char | string

Output Arguments

collapse all

Learning rate factor for the parameter, returned as a nonnegative scalar.

The software multiplies this factor by the global learning rate to determine the learning rate for the specified parameter. For example, iffactor is 2, then the learning rate for the specified parameter is twice the current global learning rate. The software determines the global learning rate based on the settings specified with the trainingOptions function.

Version History

Introduced in R2017b

expand all

Specify the path to a parameter in a networkLayer using the parameterPath argument.