leakyrelu - Apply leaky rectified linear unit activation - MATLAB (original) (raw)

Apply leaky rectified linear unit activation

Syntax

Description

The leaky rectified linear unit (ReLU) activation operation performs a nonlinear threshold operation, where any input value less than zero is multiplied by a fixed scale factor.

This operation is equivalent to

Note

This function applies the leaky ReLU operation to dlarray data. If you want to apply leaky ReLU activation within a dlnetwork object, use leakyReluLayer.

[Y](#mw%5F60c2b168-a325-4440-9399-9d974debaa3e) = leakyrelu([X](#mw%5Fc2dfd3c0-23ae-4595-a3b7-987c63934bbb%5Fsep%5Fmw%5F6f734774-5538-44ff-ad05-7e744f97893a)) computes the leaky ReLU activation of the input X by applying a threshold operation. All values in X less than zero are multiplied by a default scale factor of 0.01.

example

[Y](#mw%5F60c2b168-a325-4440-9399-9d974debaa3e) = leakyrelu([X](#mw%5Fc2dfd3c0-23ae-4595-a3b7-987c63934bbb%5Fsep%5Fmw%5F6f734774-5538-44ff-ad05-7e744f97893a),[scaleFactor](#mw%5Fe074da10-6b8b-40f6-809c-f95d7102f449)) specifies the scale factor for the leaky ReLU operation.

example

Examples

collapse all

Create a formatted dlarray object containing a batch of 128 28-by-28 images with 3 channels. Specify the format 'SSCB' (spatial, spatial, channel, batch).

miniBatchSize = 128; inputSize = [28 28]; numChannels = 3; X = rand(inputSize(1),inputSize(2),numChannels,miniBatchSize); X = dlarray(X,"SSCB");

View the size and format of the input data.

Apply the leaky ReLU operation using the leakyrelu function.

View the size and format of the output.

Create a formatted dlarray object containing a batch of 128 28-by-28 images with 3 channels. Specify the format 'SSCB' (spatial, spatial, channel, batch).

miniBatchSize = 128; inputSize = [28 28]; numChannels = 3; X = rand(inputSize(1),inputSize(2),numChannels,miniBatchSize); X = dlarray(X,"SSCB");

View the size and format of the input data.

Apply the leaky ReLU operation using the leakyrelu function and specify a scale of 0.5.

View the size and format of the output.

Input Arguments

collapse all

Input data, specified as a formatted or unformatted dlarray object.

Scale factor for negative inputs, specified as a numeric scalar. The default value is 0.01.

Data Types: single | double

Output Arguments

collapse all

Leaky ReLU activations, returned as a dlarray. The outputY has the same underlying data type as the inputX.

If the input data X is a formatted dlarray,Y has the same dimension format as X. If the input data is not a formatted dlarray, Y is an unformatted dlarray with the same dimension order as the input data.

Extended Capabilities

expand all

Usage notes and limitations:

• scaleFactor must be a constant during code generation.

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same limitations apply to GPU code generation.

The leakyrelu function supports GPU array input with these usage notes and limitations:

• When the input argument X is a gpuArray or adlarray with underlying data of type gpuArray, this function runs on the GPU.

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2019b