Estimator configuration for framework profiling (original) (raw)

Warning

In favor of Amazon SageMaker Profiler, SageMaker AI Debugger deprecates the framework profiling feature starting from TensorFlow 2.11 and PyTorch 2.0. You can still use the feature in the previous versions of the frameworks and SDKs as follows.

• SageMaker Python SDK <= v2.130.0
• PyTorch >= v1.6.0, < v2.0
• TensorFlow >= v2.3.1, < v2.11

See also March 16, 2023.

To enable Debugger framework profiling, configure theframework_profile_params parameter when you construct an estimator. Debugger framework profiling collects framework metrics, such as data from initialization stage, data loader processes, Python operators of deep learning frameworks and training scripts, detailed profiling within and between steps, with cProfile or Pyinstrument options. Using the FrameworkProfile class, you can configure custom framework profiling options.

Note

Before getting started with Debugger framework profiling, verify that the framework used to build your model is supported by Debugger for framework profiling. For more information, see Supported frameworks and algorithms.

Debugger saves the framework metrics in a default S3 bucket. The format of the default S3 bucket URI iss3://sagemaker-<region>-<12digit_account_id>/<training-job-name>/profiler-output/.

Topics

• Default framework profiling
• Default system monitoring and customized framework profiling for target steps or a target time range
• Default system monitoring and customized framework profiling with different profiling options

Configure settings for basic profiling of system resource utilization

Default profiling

Did this page help you? - Yes

Thanks for letting us know we're doing a good job!

If you've got a moment, please tell us what we did right so we can do more of it.

Did this page help you? - No

Thanks for letting us know this page needs work. We're sorry we let you down.

If you've got a moment, please tell us how we can make the documentation better.