crosschannelnorm - Cross channel square-normalize using local responses - MATLAB (original) (raw)

Cross channel square-normalize using local responses

Since R2020a

Syntax

Description

The cross-channel normalization operation uses local responses in different channels to normalize each activation. Cross-channel normalization typically follows a relu operation. Cross-channel normalization is also known as local response normalization.

[Y](#mw%5F3d7008b2-bddf-4a6c-ba3e-181f89cae4c0) = crosschannelnorm([X](#mw%5Fb3fb0884-36d4-4a68-ac10-6b3b5a6f0002),[windowSize](#mw%5Fa31f9f3d-b927-4a0d-a2ed-f9bb4723ee43)) normalizes each element of X with respect to local values in the same position in nearby channels. The normalized elements in Y are calculated from the elements in X using the following formula.

where y is an element of Y,x is the corresponding element of X,ss is the sum of the squares of the elements in the channel region defined by windowSize, and α, β, and K are hyperparameters in the normalization.

example

[Y](#mw%5F3d7008b2-bddf-4a6c-ba3e-181f89cae4c0) = crosschannelnorm([X](#mw%5Fb3fb0884-36d4-4a68-ac10-6b3b5a6f0002),[windowSize](#mw%5Fa31f9f3d-b927-4a0d-a2ed-f9bb4723ee43),'DataFormat',FMT) also specifies the dimension format FMT when X is an unformatted dlarray, in addition to the input arguments the previous syntax. The output Y is an unformatted dlarray with the same dimension order as X.

example

[Y](#mw%5F3d7008b2-bddf-4a6c-ba3e-181f89cae4c0) = crosschannelnorm(___,[Name,Value](#namevaluepairarguments)) specifies options using one or more name-value pair arguments in addition to the input arguments in previous syntaxes. For example, 'Beta',0.8 sets the value of the β contrast constant to 0.8.

example

Examples

collapse all

Normalize Data Using Values of Adjacent Channels

Use crosschannelnorm to normalize each observation of a mini-batch using values from adjacent channels.

Create the input data as ten observations of random values with a height and width of eight and six channels.

height = 8; width = 8; channels = 6; observations = 10;

X = rand(height,width,channels,observations); X = dlarray(X,'SSCB');

Compute the cross-channel normalization using a channel window size of three.

Y = crosschannelnorm(X,3);

Each value in each observation of X is normalized using the element in the previous channel and the element in the next channel.

Compare Normalized and Original Data

Values at the edges of an array are normalized using contributions from fewer channels, depending on the size of the channel window.

Create the input data as an array of ones with a height and width of two and three channels.

height = 2; width = 2; channels = 3;

X = ones(height,width,channels); dlX = dlarray(X);

Normalize the data using a channel-window size of 3, an α of 1, a β of 1, and a K of 1e-5. Specify a data format of 'SSC'.

dlY = crosschannelnorm(dlX,3,'Alpha',1,'Beta',1,'K',1e-5,'DataFormat','SSC');

Compare the values in the original and the normalized data by reshaping the three-channel arrays into 2-D matrices.

dlX = 2x6 dlarray

 1     1     1     1     1     1
 1     1     1     1     1     1

dlY = 2x6 dlarray

1.5000    1.5000    1.0000    1.0000    1.5000    1.5000
1.5000    1.5000    1.0000    1.0000    1.5000    1.5000

For the first and last channels, the sum of squares is calculated using only two values. For the middle channel, the sum of squares contains the values of all three channels.

Use Cross-Channel Normalization in a Model Function

Typically, the cross-channel normalization operation follows a ReLU operation. For example, the GoogLeNet architecture contains convolutional operations followed by ReLU and cross-channel normalization operations.

The function modelFunction defined at the end of this example shows how you can use cross-channel normalization in a model. Use modelFunction to find the grouped convolution and ReLU activation of some input data and then normalize the result using cross-channel normalization with a window size of 5.

Create the input data as a single observation of random values with a height and width of ten and four channels.

height = 10; width = 10; channels = 4; observations = 1;

X = rand(height,width,channels,observations); dlX = dlarray(X,'SSCB');

Create the parameters for the grouped convolution operation. For the weights, use a filter height and width of three, two channels per group, three filters per group, and two groups. Use a value of zero for the bias.

filterSize = [3 3]; numChannelsPerGroup = 2; numFiltersPerGroup = 3 ; numGroups = 2;

params = struct; params.conv.weights = rand(filterSize(1),filterSize(2),numChannelsPerGroup,numFiltersPerGroup,numGroups); params.conv.bias = 0;

Apply the modelFunction to the data dlX.

dlY = modelFunction(dlX,params);

function dlY = modelFunction(dlX,params)

dlY = dlconv(dlX,params.conv.weights,params.conv.bias); dlY = relu(dlY); dlY = crosschannelnorm(dlY,5);

end

Input Arguments

collapse all

X — Input data

dlarray

Input data, specified as a dlarray with or without data format. When X is an unformatted dlarray, you must specify the data format using the 'DataFormat',FMT name-value pair.

You can specify up to two dimensions in X as'S' dimensions.

Data Types: single | double

windowSize — Size of channel window

scalar integer

Size of the channel window, which controls the number of channels that are used for the normalization of each element, specified as a positive integer.

If windowSize is even, then the window is asymmetric. The software looks at the previous floor((windowSize-1)/2) channels and the following floor((windowSize)/2) channels. For example, ifwindowSize is 4, then the function normalizes each element by its neighbor in the previous channel and by its neighbors in the next two channels.

Example: 3

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32

Name-Value Arguments

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: 'Alpha',2e-4,'Beta',0.8 sets the multiplicative normalization constant to 0.0002 and the contrast constant exponent to 0.8.

DataFormat — Description of data dimensions

character vector | string scalar

Description of the data dimensions, specified as a character vector or string scalar.

A data format is a string of characters, where each character describes the type of the corresponding data dimension.

The characters are:

For example, consider an array containing a batch of sequences where the first, second, and third dimensions correspond to channels, observations, and time steps, respectively. You can specify that this array has the format "CBT" (channel, batch, time).

You can specify multiple dimensions labeled "S" or "U". You can use the labels "C", "B", and"T" once each, at most. The software ignores singleton trailing"U" dimensions after the second dimension.

If the input data is not a formatted dlarray object, then you must specify the DataFormat option.

For more information, see Deep Learning Data Formats.

Data Types: char | string

Alpha — Normalization constant (α)

1e-4 (default) | numeric scalar

Normalization constant (α) that multiplies the sum of the squared values, specified as the comma-separated pair consisting of'Alpha' and a numeric scalar. The default value is1e-4.

Example: 'Alpha',2e-4

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32

Beta — Contrast constant (β)

0.75 (default) | numeric scalar greater than or equal to 0.01

Contrast constant (β), specified as the comma-separated pair consisting of 'Beta' and a numeric scalar greater than or equal to0.01. The default value is 0.75.

Example: 'Beta',0.8

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32

K — Normalization hyperparameter (K)

2 (default) | numeric scalar greater than or equal to 1e-5

Normalization hyperparameter (K) used to avoid singularities in the normalization, specified as the comma-separated pair consisting of'K' and a numeric scalar greater than or equal to1e-5. The default value is 2.

Example: 'K',2.5

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32

Output Arguments

collapse all

Y — Normalized data

dlarray

Normalized data, returned as a dlarray. The outputY has the same underlying data type as the inputX.

If the input data X is a formatted dlarray,Y has the same dimension labels as X. If the input data is an unformatted dlarray, Y is an unformatted dlarray with the same dimension order as the input data.

More About

Extended Capabilities

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

The crosschannelnorm function supports GPU array input with these usage notes and limitations:

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2020a