SageMaker model parallelism library v2 (original) (raw)

The Amazon SageMaker AI model parallelism library is a capability of SageMaker AI that enables high performance and optimized large scale training on SageMaker AI accelerate compute instances. TheCore features of the SageMaker model parallelism library v2 include techniques and optimizations to accelerate and simplify large model training, such as hybrid sharded data parallelism, tensor parallelism, activation checkpointing, and activation offloading. You can use the SMP library to accelerate the training and fine-tuning of large language models (LLMs), large vision models (LVMs), and foundation models (FMs) with hundreds of billions of parameters.

The SageMaker model parallelism library v2 (SMP v2) aligns the library’s APIs and methods with open source PyTorch Fully Sharded Data Parallelism (FSDP), which gives you the benefit of SMP performance optimizations with minimal code changes. With SMP v2, you can improve the computational performance of training a state-of-the-art large model on SageMaker AI by bringing your PyTorch FSDP training scripts to SageMaker AI.

You can use SMP v2 for the general SageMaker Training jobs and distributed training workloads on Amazon SageMaker HyperPod clusters.

Topics

• Model parallelism concepts
• Supported frameworks and AWS Regions
• Use the SageMaker model parallelism library v2
• Core features of the SageMaker model parallelism library v2
• Amazon SageMaker AI model parallelism library v2 examples
• SageMaker distributed model parallelism best practices
• The SageMaker model parallel library v2 reference
• Release notes for the SageMaker model parallelism library
• (Archived) SageMaker model parallelism library v1.x

SMDDP release notes

Model parallelism concepts

Did this page help you? - Yes

Thanks for letting us know we're doing a good job!

If you've got a moment, please tell us what we did right so we can do more of it.

Did this page help you? - No

Thanks for letting us know this page needs work. We're sorry we let you down.

If you've got a moment, please tell us how we can make the documentation better.