Module: tf.distribute  |  TensorFlow v2.16.1 (original) (raw)

Module: tf.distribute

Stay organized with collections Save and categorize content based on your preferences.

Public API for tf._api.v2.distribute namespace

Modules

cluster_resolver module: Public API for tf._api.v2.distribute.cluster_resolver namespace

coordinator module: Public API for tf._api.v2.distribute.coordinator namespace

experimental module: Public API for tf._api.v2.distribute.experimental namespace

Classes

class CrossDeviceOps: Base class for cross-device reduction and broadcasting algorithms.

class DistributedDataset: Represents a dataset distributed among devices and machines.

class DistributedIterator: An iterator over tf.distribute.DistributedDataset.

class DistributedValues: Base class for representing distributed values.

class HierarchicalCopyAllReduce: Hierarchical copy all-reduce implementation of CrossDeviceOps.

class InputContext: A class wrapping information needed by an input function.

class InputOptions: Run options for experimental_distribute_dataset(s_from_function).

class InputReplicationMode: Replication mode for input function.

class MirroredStrategy: Synchronous training across multiple replicas on one machine.

class MultiWorkerMirroredStrategy: A distribution strategy for synchronous training on multiple workers.

class NcclAllReduce: NCCL all-reduce implementation of CrossDeviceOps.

class OneDeviceStrategy: A distribution strategy for running on a single device.

class ParameterServerStrategy: An multi-worker tf.distribute strategy with parameter servers.

class ReduceOp: Indicates how a set of values should be reduced.

class ReductionToOneDevice: A CrossDeviceOps implementation that copies values to one device to reduce.

class ReplicaContext: A class with a collection of APIs that can be called in a replica context.

class RunOptions: Run options for strategy.run.

class Server: An in-process TensorFlow server, for use in distributed training.

class Strategy: A state & compute distribution policy on a list of devices.

class StrategyExtended: Additional APIs for algorithms that need to be distribution-aware.

class TPUStrategy: Synchronous training on TPUs and TPU Pods.

Functions

experimental_set_strategy(...): Set a tf.distribute.Strategy as current without with strategy.scope().

get_replica_context(...): Returns the current tf.distribute.ReplicaContext or None.

get_strategy(...): Returns the current tf.distribute.Strategy object.

has_strategy(...): Return if there is a current non-default tf.distribute.Strategy.

in_cross_replica_context(...): Returns True if in a cross-replica context.