softmax - Apply softmax activation to channel dimension - MATLAB (original) (raw)

Apply softmax activation to channel dimension

Syntax

Description

The softmax activation operation applies the softmax function to the channel dimension of the input data.

The softmax function normalizes the value of the input data across the channel dimension such that it sums to one. You can regard the output of the softmax function as a probability distribution.

[Y](#mw%5F3f7d66f7-7b76-4d8f-997b-3c69be87a678) = softmax([X](#mw%5F7b65a642-9ead-48fc-9202-56c175f86417)) computes the softmax activation of the input X by applying the softmax transfer function to the channel dimension of the input data. All values in Y are between 0 and 1, and sum to 1. The input X must be a formatted dlarray. The outputY is a formatted dlarray with the same dimension format as X.

example

[Y](#mw%5F3f7d66f7-7b76-4d8f-997b-3c69be87a678) = softmax([X](#mw%5F7b65a642-9ead-48fc-9202-56c175f86417),'DataFormat',[FMT](#mw%5F5750e320-3fa4-4f8b-9192-ca902f1ec9b7%5Fsep%5Fmw%5F30d6528f-469b-47fb-a774-e80032eca842)) also specifies dimension format FMT when X is not a formatted dlarray. The output Y is an unformatteddlarray with the same dimension order as X.

Examples

collapse all

Use the softmax function to set all values in the input data to values between 0 and 1 that sum to1 over all channels.

Create the input classification data as two observations of random variables. The data can be in any of 10 categories.

numCategories = 10; observations = 2;

X = rand(numCategories,observations); X = dlarray(X,'CB');

Compute the softmax activation.

Y = softmax(X) totalProb = sum(Y,1)

Y =

10(C) x 2(B) dlarray

0.1151    0.0578
0.1261    0.1303
0.0579    0.1285
0.1270    0.0802
0.0959    0.1099
0.0562    0.0569
0.0673    0.0753
0.0880    0.1233
0.1328    0.1090
0.1337    0.1288

totalProb =

1(C) x 2(B) dlarray

1.0000    1.0000

All values in Y range between 0 and1. The values over all channels sum to 1 for each observation.

Input Arguments

collapse all

Input data, specified as a formatted dlarray or an unformatteddlarray. When X is not a formatteddlarray, you must specify the dimension label format using'DataFormat',FMT.

X must contain a 'C' channel dimension.

Data Types: single | double

Description of the data dimensions, specified as a character vector or string scalar.

A data format is a string of characters, where each character describes the type of the corresponding data dimension.

The characters are:

For example, consider an array that represents a batch of sequences where the first, second, and third dimensions correspond to channels, observations, and time steps, respectively. You can describe the data as having the format "CBT" (channel, batch, time).

You can specify multiple dimensions labeled "S" or "U". You can use the labels "C", "B", and"T" once each, at most. The software ignores singleton trailing"U" dimensions after the second dimension.

If the input data is not a formatted dlarray object, then you must specify the FMT option.

For more information, see Deep Learning Data Formats.

Data Types: char | string

Output Arguments

collapse all

Softmax activations, returned as a dlarray. All values inY are between 0 and 1. The output Y has the same underlying data type as the inputX.

If the input data X is a formatted dlarray,Y has the same dimension format as X. If the input data is not a formatted dlarray, Y is an unformatted dlarray with the same dimension order as the input data.

Algorithms

collapse all

The softmax function normalizes the input across the channel dimension, such that it sums to one. For more information, see the definition ofSoftmax Layer on the softmaxLayer reference page.

Most deep learning networks and functions operate on different dimensions of the input data in different ways.

For example, an LSTM operation iterates over the time dimension of the input data, and a batch normalization operation normalizes over the batch dimension of the input data.

To provide input data with labeled dimensions or input data with additional layout information, you can use data formats.

A data format is a string of characters, where each character describes the type of the corresponding data dimension.

The characters are:

For example, consider an array that represents a batch of sequences where the first, second, and third dimensions correspond to channels, observations, and time steps, respectively. You can describe the data as having the format "CBT" (channel, batch, time).

To create formatted input data, create a dlarray object and specify the format using the second argument.

To provide additional layout information with unformatted data, specify the format using the FMT argument.

For more information, see Deep Learning Data Formats.

Extended Capabilities

expand all

The softmax function supports GPU array input with these usage notes and limitations:

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2019b