nki.language.mish — AWS Neuron Documentation (original) (raw)

This document is relevant for: Inf2, Trn1, Trn2

nki.language.mish#

nki.language.mish(x, *, dtype=None, mask=None, **kwargs)[source]#

Mish activation function on the input, element-wise.

Mish: A Self Regularized Non-Monotonic Neural Activation Function is defined as:

\[mish(x) = x * tanh(softplus(x))\]

see: https://arxiv.org/abs/1908.08681

Parameters:

• x – a tile.
• dtype – (optional) data type to cast the output type to (see Supported Data Types for more information); if not specified, it will default to be the same as the data type of the input tile.
• mask – (optional) a compile-time constant predicate that controls whether/how this instruction is executed (see NKI API Masking for details)

Returns:

a tile that has mish of x.

This document is relevant for: Inf2, Trn1, Trn2