FileLogger (original) (raw)

Back to top

Edit this page

Toggle table of contents sidebar

class composer.loggers.FileLogger(filename='{run_name}/logs-rank{rank}.txt', remote_file_name=None, *, capture_stdout=True, capture_stderr=True, buffer_size=1, log_traces=True, flush_interval=100, overwrite=False)[source]#

Log data to a file.

Example usage:

from composer.loggers import FileLogger from composer.trainer import Trainer file_logger = FileLogger( filename="{run_name}/logs-rank{rank}.txt", buffer_size=1, flush_interval=50 ) trainer = Trainer( ..., loggers=[file_logger] )

Example output:

[FIT][step=2]: { "logged_metric": "logged_value", } [EPOCH][step=2]: { "logged_metric": "logged_value", } [BATCH][step=2]: { "logged_metric": "logged_value", } [EPOCH][step=3]: { "logged_metric": "logged_value", }

Parameters

• filename (str, optional) –
  Format string for the filename.
  The following format variables are available:
  Note
  When training with multiple devices (i.e. GPUs), ensure that '{rank}' appears in the format. Otherwise, multiple processes may attempt to write to the same file.
  Consider the following example when using default value of ‘{run_name}/logs-rank{rank}.txt’:

  file_logger = FileLogger(filename='{run_name}/logs-rank{rank}.txt')
  trainer = Trainer(loggers=[file_logger], run_name='my-awesome-run')
  file_logger.filename
  'my-awesome-run/logs-rank0.txt'
  Default: ‘{run_name}/logs-rank{rank}.txt’

• remote_file_name (str, optional) –
  Format string for the logfile’s name.
  The logfile will be periodically logged (according to the flush_interval) as a file. The file name will be determined by this format string.
  The same format variables for filename are available. Setting this parameter to None(the default) will use the same format string as filename. It is sometimes helpful to deviate from this default. For example, when filename contains an absolute path, it is recommended to set this parameter explicitely, so the absolute path does not appear in any remote file stores.
  Leading slashes ('/') will be stripped.
  Default: None (which uses the same format string as filename)
• capture_stdout (bool, optional) – Whether to include the stdout``in ``filename. (default: True)
• capture_stderr (bool, optional) – Whether to include the stderr``in ``filename. (default: True)
• buffer_size (int, optional) – Buffer size. See open(). Default: 1 for line buffering.
• log_traces (bool, optional) – Whether to log algorithm traces. See Engine for more detail.
• flush_interval (int, optional) – How frequently to flush the log to the file in batches Default: 100.
• overwrite (bool, optional) – Whether to overwrite an existing logfile. (default: False)

property filename#

The filename for the logfile.

property remote_file_name#

The remote file name for the logfile.

write(prefix, s)[source]#

Write to the logfile.

Note

If the write occurs before the Event.INIT event, the write will be enqueued, as the file is not yet open.

Parameters

• prefix (str) – A prefix for each line in the logfile.
• s (str) – The string to write. Each line will be prefixed with prefix.