DropoutLayer - Dropout layer - MATLAB (original) (raw)

Description

A dropout layer randomly sets input elements to zero with a given probability.

Creation

Syntax

Description

`layer` = dropoutLayer creates a dropout layer.

`layer` = dropoutLayer(`probability`) creates a dropout layer and sets the Probability property.

`layer` = dropoutLayer(___,'Name',`Name`) sets the optional Name property using a name-value pair and any of the arguments in the previous syntaxes. For example,dropoutLayer(0.4,'Name','drop1') creates a dropout layer with dropout probability 0.4 and name 'drop1'. Enclose the property name in single quotes.

example

Properties

expand all

Dropout

Probability for dropping out input elements, specified as a nonnegative number less than 1.

At training time, the layer randomly sets input elements to zero given by the dropout mask rand(size(X))<Probability, where X is the layer input and then scales the remaining elements by 1/(1-Probability). This operation effectively changes the underlying network architecture between iterations and helps prevent the network from overfitting [1], [2]. A higher number results in more elements being dropped during training. At prediction time, the output of the layer is equal to its input.

For image input, the layer applies a different mask for each channel of each image. For sequence input, the layer applies a different dropout mask for each time step of each sequence.

Example: 0.4

Layer

Data Types: char | string

This property is read-only.

Number of inputs to the layer, stored as 1. This layer accepts a single input only.

Data Types: double

This property is read-only.

Input names, stored as {'in'}. This layer accepts a single input only.

Data Types: cell

This property is read-only.

Number of outputs from the layer, stored as 1. This layer has a single output only.

Data Types: double

This property is read-only.

Output names, stored as {'out'}. This layer has a single output only.

Data Types: cell

Examples

collapse all

Create a dropout layer with name 'drop1'.

layer = dropoutLayer('Name','drop1')

layer = DropoutLayer with properties:

       Name: 'drop1'

Hyperparameters Probability: 0.5000

Include a dropout layer in a Layer array.

layers = [ ... imageInputLayer([28 28 1]) convolution2dLayer(5,20) reluLayer dropoutLayer fullyConnectedLayer(10) softmaxLayer]

layers = 6×1 Layer array with layers:

 1   ''   Image Input       28×28×1 images with 'zerocenter' normalization
 2   ''   2-D Convolution   20 5×5 convolutions with stride [1  1] and padding [0  0  0  0]
 3   ''   ReLU              ReLU
 4   ''   Dropout           50% dropout
 5   ''   Fully Connected   10 fully connected layer
 6   ''   Softmax           softmax

Algorithms

expand all

A dropout layer randomly sets input elements to zero with a given probability.

At training time, the layer randomly sets input elements to zero given by the dropout mask rand(size(X))<Probability, where X is the layer input and then scales the remaining elements by 1/(1-Probability). This operation effectively changes the underlying network architecture between iterations and helps prevent the network from overfitting [1], [2]. A higher number results in more elements being dropped during training. At prediction time, the output of the layer is equal to its input.

Similar to max or average pooling layers, no learning takes place in this layer.

For image input, the layer applies a different mask for each channel of each image. For sequence input, the layer applies a different dropout mask for each time step of each sequence.

Layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters in which each character describes the corresponding dimension of the data. The format consists of one or more of these characters:

For example, you can describe 2-D image data that is represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, as having the format "SSCB" (spatial, spatial, channel, batch).

DropoutLayer objects apply an element-wise operation and support input data of any format. The layer does not add or remove any dimensions, so it outputs data with the same format as its input data.

DropoutLayer objects support complex-valued input and outputs. (since R2024a) The layer applies the same underlying operation to complex-valued input as it does to real-valued input and outputs complex-valued data where applicable.

References

[1] Srivastava, Nitish, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. "Dropout: A Simple Way to Prevent Neural Networks from Overfitting." The Journal of Machine Learning Research 15, no. 1 (January 1, 2014): 1929–58

[2] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks."Communications of the ACM 60, no. 6 (May 24, 2017): 84–90. https://doi.org/10.1145/3065386.

Extended Capabilities

expand all

Usage notes and limitations:

For code generation, you must pass a dlarray object with a channel (C) dimension as the input to this layer. For example, code generation supports data format such as "SSC" or "SSCBT".

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same limitations apply to GPU code generation.

Version History

Introduced in R2016a

expand all

DropoutLayer objects support complex-valued input and outputs. The layer applies the same underlying operation to complex-valued input as it does to real-valued input and outputs complex-valued data where applicable.