composer.algorithms (original) (raw)

Back to top

Edit this page

Toggle table of contents sidebar

Efficiency methods for training.

Examples include LabelSmoothing and adding SqueezeExcite blocks, among many others.

Algorithms are implemented in both a standalone functional form (see composer.functional) and as subclasses of Algorithm for integration in the Composer Trainer. The former are easier to integrate piecemeal into an existing codebase. The latter are easier to compose together, since they all have the same public interface and work automatically with the Composer Trainer.

For ease of composability, algorithms in our Trainer are based on the two-way callbacks concept fromHoward et al, 2020. Each algorithm implements two methods:

• Algorithm.match(): returns True if the algorithm should be run given the currentState and Event.
• Algorithm.apply(): performs an in-place modification of the givenState

For example, a simple algorithm that shortens training:

from composer import Algorithm, State, Event, Logger

class ShortenTraining(Algorithm):

def match(self, state: State, event: Event, logger: Logger) -> bool:
    return event == Event.INIT

def apply(self, state: State, event: Event, logger: Logger):
    state.max_duration /= 2  # cut training time in half

For more information about events, see Event.

Classes