LabelSmoothing (original) (raw)

Back to top

Edit this page

Toggle table of contents sidebar

class composer.algorithms.LabelSmoothing(smoothing=0.1, target_key=1)[source]#

Shrink targets towards a uniform distribution as in Szegedy et al.

The smoothed labels are computed as (1 - smoothing) * targets + smoothing * unifwhere unif is a vector with elements all equal to 1 / num_classes.

Parameters

• smoothing – Strength of the label smoothing, in \([0, 1]\).smoothing=0 means no label smoothing, andsmoothing=1 means maximal smoothing (targets are ignored). Default: 0.1.
• target_key (str | int | tuple[ Callable , Callable ] | Any , optional) – A key that indexes to the target from the batch. Can also be a pair of get and set functions, where the getter is assumed to be first in the pair. The default is 1, which corresponds to any sequence, where the second element is the target. Default: 1.

Example

from composer.algorithms import LabelSmoothing algorithm = LabelSmoothing(smoothing=0.1) trainer = Trainer( model=model, train_dataloader=train_dataloader, eval_dataloader=eval_dataloader, max_duration="1ep", algorithms=[algorithm], optimizers=[optimizer] )