Batch Normalization Layer - Batch normalization layer - Simulink (original) (raw)

Batch normalization layer

Since R2024b

Libraries:
Deep Learning Toolbox / Deep Learning Layers / Normalization Layers

Description

The Batch Normalization Layer block normalizes input data across all observations for each channel independently. To speed up training of the convolutional neural network and reduce the sensitivity to network initialization, use batch normalization layers between convolutional layers and nonlinearities, such as ReLU layers.

After normalization, the layer scales the input with a learnable scale factor_γ_ and shifts it by a learnable offset_β_.

This block accepts data that has dimensions corresponding to the format that you specify with the Data format block parameter.

The exportNetworkToSimulink function generates this block to represent a batchNormalizationLayer object.

Ports

Input

expand all

Input data to normalize. The data must have dimensions corresponding to the format that you specify with the Data format block parameter.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | fixed point

Output

expand all

Normalized output data. The output data has dimensions corresponding to the format that you specify with the Data format block parameter.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | fixed point

Parameters

expand all

To edit block parameters interactively, use theProperty Inspector. From the Simulink® Toolstrip, on the Simulation tab, in thePrepare gallery, select Property Inspector.

Main

Specify the name of a workspace variable that contains abatchNormalizationLayer object from a trained network. TheBatch Normalization Layer block configures itself by using the properties of the object and calculates the block output by using the learnable parameters of the object.

Programmatic Use

Data format for the input data. The options use the same notation as the fmt argument of thedlarray object, except layer blocks do not support the Batch (B) dimension and instead assume an observation number of1.

Programmatic Use

Data Types

Minimum value of the output range that the software checks.

The software uses the minimum value to perform:

• Parameter range checking for some blocks. For more information, see Specify Minimum and Maximum Values for Block Parameters (Simulink).
• Simulation range checking. For more information, see Specify Signal Ranges (Simulink) andEnable Simulation Range Checking (Simulink).
• Automatic scaling of fixed-point data types.
• Optimization of the code that you generate from the model. This optimization can remove algorithmic code and affect the results of some simulation modes, such as SIL or external mode. For more information, seeOptimize using the specified minimum and maximum values (Embedded Coder).

Tips

Output minimum does not saturate or clip the actual output signal. Use the Saturation (Simulink) block instead.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Maximum value of the output range that the software checks.

The software uses the maximum value to perform:

• Parameter range checking for some blocks. For more information, see Specify Minimum and Maximum Values for Block Parameters (Simulink).
• Simulation range checking. For more information, see Specify Signal Ranges (Simulink) andEnable Simulation Range Checking (Simulink).
• Automatic scaling of fixed-point data types.
• Optimization of the code that you generate from the model. This optimization can remove algorithmic code and affect the results of some simulation modes, such as SIL or external mode. For more information, seeOptimize using the specified minimum and maximum values (Embedded Coder).

Tips

Output maximum does not saturate or clip the actual output signal. Use the Saturation (Simulink) block instead.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Choose the data type for the output. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType. When you select Inherit: Inherit via internal rule, Simulink chooses a data type to balance numerical accuracy, performance, and generated code size, while taking into account the properties of the embedded target hardware.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Select this parameter to prevent the fixed-point tools from overriding the data types you specify on this block. For more information, see Lock the Output Data Type Setting (Fixed-Point Designer).

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Specify the rounding mode for fixed-point operations. For more information, see Rounding Modes (Fixed-Point Designer).

Block parameters always round to the nearest representable value. To control the rounding of a block parameter, enter an expression using a MATLAB® rounding function in the mask field.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Specify whether integer overflows saturate or wrap.

• on — Overflows saturate to either the minimum or maximum value that the data type can represent.
• off — Overflows wrap to the appropriate value that the data type can represent.

For example, the maximum value that the signed 8-bit integer int8 can represent is 127. Any block operation result greater than the maximum value causes overflow of the 8-bit integer.

• With this parameter selected, the block output saturates at 127. Similarly, the block output saturates at a minimum output value of –128.
• With this parameter cleared, the software interprets the overflow-causing value as int8, which can produce an unintended result. For example, a block result of 130 (binary 1000 0010) expressed asint8 is –126.

Tips

• Set this parameter to on when your model has a possible overflow and you want explicit saturation protection in the generated code.
• To optimize the efficiency of your generated code, keep the defaultoff setting for this parameter. Using the default setting also helps you avoid overspecifying how the block handles out-of-range signals. For more information, see Troubleshoot Signal Range Errors (Simulink).
• When you select this parameter, saturation applies to every internal operation on the block, not just the output or the result.
• In general, the code generation process can detect when overflow is not possible. In this case, the code generator does not produce saturation code.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Choose the data type for the output of the EffectiveScale Constant block inside the Batch Normalization Layer block. The Constant block uses the Scale,TrainedVariance, and Epsilon properties of the object that you specify with the Layer parameter to calculate the effective scale. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Choose the data type for the output of the EffectiveOffset Constant block inside the Batch Normalization Layer block. The Constant block uses the Offset,TrainedMean, Scale,TrainedVariance, and Epsilon properties of the object that you specify with the Layer parameter to calculate the effective offset.The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Choose the data type of the outputs of the Addition andProduct blocks inside the BatchNorm For Each Subsystem block inside the Batch Normalization Layer block. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType. When you select Inherit: Inherit via internal rule, Simulink chooses a data type to balance numerical accuracy, performance, and generated code size, while taking into account the properties of the embedded target hardware.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Execution

Specify the discrete interval between sample time hits or specify another type of sample time, such as continuous (0) or inherited (-1). For more options, see Types of Sample Time (Simulink).

By default, the block inherits its sample time based on the context of the block within the model.

Programmatic Use

To set the block parameter value programmatically, use the set_param (Simulink) function.

Extended Capabilities

Version History

Introduced in R2024b

expand all

The Batch Normalization Layer block now calculates effective scale and offset values outside the BatchNorm For Each Subsystem block by using two Constant blocks that reference the Offset, TrainedMean,Scale, TrainedVariance, andEpsilon properties of the object that you specify with theLayer parameter. As a result, the Trained mean, Standard deviation, Scale, and Offset parameters are replaced by Effective scale and Effective offset parameters that cast the data types of the outputs of the new Constant blocks. TheAccumulator parameter now casts the data types of the outputs of aProduct block and an Addition block inside theBatchNorm subsystem.

Starting in R2025a, the default value for the Layer parameter is 'layerObject'. In previous versions, the default value is 'layer'. If you have code that programmatically creates Simulink and relies on variables with the name 'layer', update your code so that the variable has the name 'layerObject'.