SoftmaxLayer - Softmax layer - MATLAB (original) (raw)

Description

A softmax layer applies a softmax function to the input.

Creation

Syntax

Description

`layer` = softmaxLayer creates a softmax layer.

`layer` = softmaxLayer(Name=name) creates a softmax layer and sets the optional Name property using a name-value pair. For example,softmaxLayer(Name="sm1") creates a softmax layer with the name "sm1".

example

Properties

expand all

Data Types: char | string

This property is read-only.

Number of inputs to the layer, stored as 1. This layer accepts a single input only.

Data Types: double

This property is read-only.

Input names, stored as {'in'}. This layer accepts a single input only.

Data Types: cell

This property is read-only.

Number of outputs from the layer, stored as 1. This layer has a single output only.

Data Types: double

This property is read-only.

Output names, stored as {'out'}. This layer has a single output only.

Data Types: cell

Examples

collapse all

Create a softmax layer with the name "sm1".

layer = softmaxLayer(Name="sm1")

layer = SoftmaxLayer with properties:

Name: 'sm1'

Include a softmax layer in a Layer array.

layers = [ ... imageInputLayer([28 28 1]) convolution2dLayer(5,20) reluLayer maxPooling2dLayer(2,Stride=2) fullyConnectedLayer(10) softmaxLayer]

layers = 6×1 Layer array with layers:

 1   ''   Image Input       28×28×1 images with 'zerocenter' normalization
 2   ''   2-D Convolution   20 5×5 convolutions with stride [1  1] and padding [0  0  0  0]
 3   ''   ReLU              ReLU
 4   ''   2-D Max Pooling   2×2 max pooling with stride [2  2] and padding [0  0  0  0]
 5   ''   Fully Connected   10 fully connected layer
 6   ''   Softmax           softmax

Algorithms

expand all

A softmax layer applies a softmax function to the input.

For classification problems, a softmax layer and then a classification layer usually follow the final fully connected layer.

The output unit activation function is the softmax function:

where 0≤yr≤1 and ∑j=1kyj=1.

The softmax function is the output unit activation function after the last fully connected layer for multi-class classification problems:

where 0≤P(cr|x,θ)≤1 and ∑j=1kP(cj|x,θ)=1. Moreover, ar=ln(P(x,θ|cr)P(cr)), P(x,θ|cr) is the conditional probability of the sample given class r, and P(cr) is the class prior probability.

The softmax function is also known as the normalized exponential and can be considered the multi-class generalization of the logistic sigmoid function [1].

Layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters in which each character describes the corresponding dimension of the data. The format consists of one or more of these characters:

• "S" — Spatial
• "C" — Channel
• "B" — Batch
• "T" — Time
• "U" — Unspecified

For example, you can describe 2-D image data that is represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, as having the format "SSCB" (spatial, spatial, channel, batch).

You can interact with these dlarray objects in automatic differentiation workflows, such as those for developing a custom layer, using a functionLayer object, or using the forward and predict functions withdlnetwork objects.

This table shows the supported input formats of SoftmaxLayer objects and the corresponding output format. If the software passes the output of the layer to a custom layer that does not inherit from the nnet.layer.Formattable class, or aFunctionLayer object with the Formattable property set to 0 (false), then the layer receives an unformatted dlarray object with dimensions ordered according to the formats in this table. The formats listed here are only a subset. The layer may support additional formats such as formats with additional "S" (spatial) or"U" (unspecified) dimensions.

In dlnetwork objects, SoftmaxLayer objects also support these input and output format combinations.

References

[1] Bishop, C. M. Pattern Recognition and Machine Learning. Springer, New York, NY, 2006.

Extended Capabilities

Version History

Introduced in R2016a