LoggerDestination (original) (raw)
Toggle table of contents sidebar
class composer.loggers.LoggerDestination(*args, **kwargs)[source]#
Base class for logger destination.
As this class extends Callback, logger destinations can run on any training loopEvent. For example, it may be helpful to run onEvent.EPOCH_END to perform any flushing at the end of every epoch.
Example
from composer.loggers import LoggerDestination from composer.trainer import Trainer class MyLogger(LoggerDestination): ... def log_hyperparameters(self, data): ... print(f'Batch {int(state.timestamp.batch)}: {data}') logger = MyLogger() trainer = Trainer( ... ..., ... loggers=[logger] ... ) Batch 0: {'composer_version': ...} Batch 0: {'composer_commit_hash': ...} Batch 0: {'num_nodes': ...} Batch 0: {'rank_zero_seed': ...}
Indicates whether LoggerDestination can upload files.
Defaults to false, should return True for derived logger classes that implement upload_file().
Returns
bool – Whether the class supports uploading files.
download_file(remote_file_name, destination, overwrite=False, progress_bar=True)[source]#
Handle downloading a file named remote_file_name to destination.
Parameters
- remote_file_name (str) – The name of the file.
- destination (str) – The destination filepath.
- overwrite (bool) – Whether to overwrite an existing file at
destination. Defaults toFalse. - progress_bar (bool, optional) – Whether to show a progress bar. Ignored if
pathis a local file. (default:True)
log_hyperparameters(hyperparameters)[source]#
Log hyperparameters, configurations, and settings.
Logs any parameter/configuration/setting that doesn’t vary during the run.
Parameters
hyperparameters (dict[_str,_ Any ]) – A dictionary mapping hyperparameter names (strings) to their values (Any).
log_images(images, name='Images', channels_last=False, step=None, masks=None, mask_class_labels=None, use_table=True)[source]#
Log images. Logs any tensors or arrays as images.
Parameters
- images (np.ndarray | Tensor | Sequence _[_ _np.ndarray_ _|_ Tensor]) – Dictionary mapping image(s)’ names (str) to an image of array of images.
- name (str) – The name of the image(s). (Default:
'Images') - channels_last (bool) – Whether the channel dimension is first or last. (Default:
False) - step (Optional _[_int] , optional) – The current step or batch of training at the time of logging. Defaults to None. If not specified the specific LoggerDestination implementation will choose a step (usually a running counter).
- masks (dict[_str,_ np.ndarray | Tensor | Sequence _[_ _np.ndarray_ _|_ Tensor] ] , optional) – A dictionary mapping the mask name (e.g. predictions or ground truth) to a sequence of masks.
- mask_class_labels (dict[_int,_ str] , optional) – Dictionary mapping label id to its name. Used for labelling each color in the mask.
- use_table (bool) – Whether to make a table of the images or not. (default:
True). Only for use with WandB.
log_metrics(metrics, step=None)[source]#
Log metrics or parameters that vary during training.
Parameters
- metrics (dict[_str,_ float]) – Dictionary mapping metric name (str) to metric scalar value (float)
- step (Optional _[_int] , optional) – The current step or batch of training at the time of logging. Defaults to None. If not specified the specific LoggerDestination implementation will choose a step (usually a running counter).
log_table(columns, rows, name='Table', step=None)[source]#
Log a table.
Parameters
- columns (list_[_str]) – Names of the columns in the table.
- rows (list_[_list[ Any ] ]) – 2D row-oriented array of values.
- name (str) – Name of table. (Default:
'Table') - step (Optional _[_int] , optional) – The current step or batch of training at the time of logging. Defaults to None. If not specified the specific LoggerDestination implementation will choose a step (usually a running counter).
Log traces. Logs any debug-related data like algorithm traces.
Parameters
traces (dict[_str,_ float]) – Dictionary mapping trace names (str) to trace (Any).
upload_file(state, remote_file_name, file_path, *, overwrite)[source]#
Handle uploading a file stored at file_path to a file named remote_file_name.
Subclasses should implement this method to store logged files (e.g. copy it to another folder or upload it to an object store). However, not all loggers need to implement this method. For example, the TQDMLogger does not implement this method, as it cannot handle file uploads.
Note
- This method will block the training loop. For optimal performance, it is recommended that this method copy the file to a temporary directory, enqueue the copied file for processing, and return. Then, use a background thread(s) or process(s) to read from this queue to perform any I/O.
- After this method returns, training can resume, and the contents of
file_pathmay change (or be may deleted). Thus, if processing the file in the background (as is recommended), it is necessary to first copy the file to a temporary directory. Otherwise, the original file may no longer exist, or the logged file can be corrupted (e.g., if the logger destination is reading from file while the training loop is writing to it).
Parameters