leakyrelu - Apply leaky rectified linear unit activation - MATLAB (original) (raw)

Main Content

Apply leaky rectified linear unit activation

Syntax

Description

The leaky rectified linear unit (ReLU) activation operation performs a nonlinear threshold operation, where any input value less than zero is multiplied by a fixed scale factor.

This operation is equivalent to

Note

This function applies the leaky ReLU operation to dlarray data. If you want to apply leaky ReLU activation within a dlnetwork object, use leakyReluLayer.

[Y](#mw%5F60c2b168-a325-4440-9399-9d974debaa3e) = leakyrelu([X](#mw%5Fc2dfd3c0-23ae-4595-a3b7-987c63934bbb%5Fsep%5Fmw%5F6f734774-5538-44ff-ad05-7e744f97893a)) computes the leaky ReLU activation of the input X by applying a threshold operation. All values in X less than zero are multiplied by a default scale factor of 0.01.

example

[Y](#mw%5F60c2b168-a325-4440-9399-9d974debaa3e) = leakyrelu([X](#mw%5Fc2dfd3c0-23ae-4595-a3b7-987c63934bbb%5Fsep%5Fmw%5F6f734774-5538-44ff-ad05-7e744f97893a),[scaleFactor](#mw%5Fe074da10-6b8b-40f6-809c-f95d7102f449)) specifies the scale factor for the leaky ReLU operation.

example

Examples

collapse all

Apply Leaky ReLU Operation

Create a formatted dlarray object containing a batch of 128 28-by-28 images with 3 channels. Specify the format 'SSCB' (spatial, spatial, channel, batch).

miniBatchSize = 128; inputSize = [28 28]; numChannels = 3; X = rand(inputSize(1),inputSize(2),numChannels,miniBatchSize); X = dlarray(X,"SSCB");

View the size and format of the input data.

Apply the leaky ReLU operation using the leakyrelu function.

View the size and format of the output.

Specify Leaky ReLU Scale

Create a formatted dlarray object containing a batch of 128 28-by-28 images with 3 channels. Specify the format 'SSCB' (spatial, spatial, channel, batch).

miniBatchSize = 128; inputSize = [28 28]; numChannels = 3; X = rand(inputSize(1),inputSize(2),numChannels,miniBatchSize); X = dlarray(X,"SSCB");

View the size and format of the input data.

Apply the leaky ReLU operation using the leakyrelu function and specify a scale of 0.5.

View the size and format of the output.

Input Arguments

collapse all

X — Input data

dlarray object

Input data, specified as a formatted or unformatted dlarray object.

scaleFactor — Scale factor for negative inputs

0.01 (default) | numeric scalar

Scale factor for negative inputs, specified as a numeric scalar. The default value is 0.01.

Data Types: single | double

Output Arguments

collapse all

Y — Leaky ReLU activations

dlarray

Leaky ReLU activations, returned as a dlarray. The outputY has the same underlying data type as the inputX.

If the input data X is a formatted dlarray,Y has the same dimension format as X. If the input data is not a formatted dlarray, Y is an unformatted dlarray with the same dimension order as the input data.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same limitations apply to GPU code generation.

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

The leakyrelu function supports GPU array input with these usage notes and limitations:

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2019b