Model Builder — sagemaker 2.247.0 documentation (original) (raw)

sagemaker

This module contains classes related to Amazon Sagemaker Model Builder

class sagemaker.serve.ModelBuilder(model_path='/tmp/sagemaker/model-builder/b231354248a611f0a2437e51270081f6', role_arn=None, sagemaker_session=None, name=, mode=Mode.SAGEMAKER_ENDPOINT, shared_libs=, dependencies=, env_vars=, log_level=10, content_type=None, accept_type=None, s3_model_data_url=None, instance_type='ml.c5.xlarge', schema_builder=None, model=None, inference_spec=None, image_uri=None, image_config=None, vpc_config=None, model_server=None, model_metadata=None, inference_component_name=None, modelbuilder_list=None, resource_requirements=None)

Class that builds a deployable model.

Parameters:

• role_arn (Optional _[_str]) – The role for the endpoint.
• model_path (Optional _[_str]) – The path of the model directory.
• sagemaker_session (Optional _[_sagemaker.session.Session]) – The SageMaker session to use for the execution.
• name (Optional _[_str]) – The model name.
• mode (Optional [ Mode ]) –
  The mode of operation. The following modes are supported:
  • Mode.SAGEMAKER_ENDPOINT: Launch on a SageMaker endpoint
  • Mode.LOCAL_CONTAINER: Launch locally with a container
  • Mode.IN_PROCESS: Launch locally to a FastAPI server instead of using a container.
• shared_libs (List _[_str]) – Any shared libraries you want to bring into the model packaging.
• dependencies (Optional [_ _Dict_ _[_str,_ Any ]) –
  The dependencies of the model or container. Takes a dict as an input where you can specify autocapture as True or False, a requirements file, or custom dependencies as a list. A sample dependencies dict:
  {
  "auto": False,
  "requirements": "/path/to/requirements.txt",
  "custom": ["custom_module==1.2.3",
  "other_module@http://some/website.whl"],

}

• env_vars (Optional [_ _Dict_ _[_str,_ str]) – The environment variables for the runtime execution.
• log_level (Optional _[_int]) – The log level. Possible values are CRITICAL,ERROR, WARNING, INFO, DEBUG, and NOTSET.
• content_type (Optional _[_str]) – The content type of the endpoint input data. This value is automatically derived from the input sample, but you can override it.
• accept_type (Optional _[_str]) – The content type of the data accepted from the endpoint. This value is automatically derived from the output sample, but you can override the value.
• s3_model_data_url (Optional _[_str]) – The S3 location where you want to upload the model package. Defaults to a S3 bucket prefixed with the account ID.
• instance_type (Optional _[_str]) – The instance type of the endpoint. Defaults to the CPU instance type to help narrow the container type based on the instance family.
• schema_builder (Optional _[_SchemaBuilder]) – The input/output schema of the model. The schema builder translates the input into bytes and converts the response into a stream. All translations between the server and the client are handled automatically with the specified input and output. The schema builder can be omitted for HuggingFace models with task types TextGeneration, TextClassification, and QuestionAnswering. Omitting SchemaBuilder is in beta for FillMask, and AutomaticSpeechRecognition use-cases.
• model (Optional [_ _Union_ _[_object,_ str, ModelTrainer, TrainingJob , Estimator] ]) – Define object from which training artifacts can be extracted. Either model or inference_specis required for the model builder to build the artifact.
• inference_spec (InferenceSpec) – The inference spec file with your customizedinvoke and load functions.
• image_uri (Optional _[_str]) – The container image uri (which is derived from a SageMaker-based container).
• image_config (dict[_str,_ str] or dict[_str,_ PipelineVariable]) – Specifies whether the image of model container is pulled from ECR, or private registry in your VPC. By default it is set to pull model container image from ECR. (default: None).
• vpc_config (Optional [_ _Dict_ _[_str,_ List [_ _Union_ _[_str,_ PipelineVariable] ] ] ]) – The VpcConfig set on the model (default: None) * ‘Subnets’ (List[Union[str, PipelineVariable]]): List of subnet ids. * ‘SecurityGroupIds’ (List[Union[str, PipelineVariable]]]): List of security group ids.
• model_server (Optional [ ModelServer ]) – The model server to which to deploy. You need to provide this argument when you specify an image_uriin order for model builder to build the artifacts correctly (according to the model server). Possible values for this argument areTORCHSERVE, MMS, TENSORFLOW_SERVING, DJL_SERVING,TRITON, TGI, and TEI.
• model_metadata (Optional [_ _Dict_ _[_str,_ Any ]) – Dictionary used to override model metadata. Currently, HF_TASK is overridable for HuggingFace model. HF_TASK should be set for new models without task metadata in the Hub, adding unsupported task types will throw an exception. MLFLOW_MODEL_PATH is available for providing local path or s3 path to MLflow artifacts. However, MLFLOW_MODEL_PATH is experimental and is not intended for production use at this moment. CUSTOM_MODEL_PATH is available for providing local path or s3 path to model artifacts. FINE_TUNING_MODEL_PATH is available for providing s3 path to fine-tuned model artifacts. FINE_TUNING_JOB_NAMEis available for providing fine-tuned job name. Both FINE_TUNING_MODEL_PATH andFINE_TUNING_JOB_NAME are mutually exclusive.
• inference_component_name (Optional _[_str]) – The name for an inference component created from this ModelBuilder instance. This or resource_requirements must be set to denote that this instance refers to an inference component.
• modelbuilder_list (List_[_ModelBuilder] | None) – Optional[List[ModelBuilder]] = List of ModelBuilder objects which can be built in bulk and subsequently deployed in bulk. Currently only supports deployments for inference components.
• resource_requirements (ResourceRequirements | None) – Optional[ResourceRequirements] = Defines the compute resources allocated to run the model assigned to the inference component. This orinference_component_name must be set to denote that this instance refers to an inference component. If inference_component_name is set but this is not and a JumpStart model ID is specified, pre-benchmarked deployment configs will attempt to be retrieved for the model.

ModelBuilder.build(mode=None, role_arn=None, sagemaker_session=None)

Creates deployable Model instances with all provided ModelBuilder objects.

Parameters:

• mode (Type [ Mode ] , optional) – The mode. Defaults to None.
• role_arn (str, optional) – The IAM role arn. Defaults to None.
• sagemaker_session (Optional _[_Session]) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the function creates one using the default AWS configuration chain.

Returns:

A deployable ModelBuilder object if multipleModelBuilders were built, or a deployable Model object.

Return type:

Union[ModelBuilder, Type[Model]]

ModelBuilder.save(*args, **kwargs)

class sagemaker.serve.InferenceSpec

Abstract base class for holding custom load, invoke and prepare functions.

Provides a skeleton for customization to override the methodsload, invoke and prepare.

class sagemaker.serve.SchemaBuilder(sample_input, sample_output, input_translator=None, output_translator=None)

Automatically detects the serializer and deserializer for your model.

This is done by inspecting the sample_input and sample_output object. Alternatively, provide your custom serializer and deserializer for your request or response by creating a class that inheritsCustomPayloadTranslator and provide it to SchemaBuilder.

Parameters:

• sample_input (object) – Sample input to the model which can be used for testing. The schema builder internally generates the content type and corresponding serializing functions.
• sample_output (object) – Sample output to the model which can be used for testing. The schema builder internally generates the accept type and corresponding serializing functions.
• input_translator (Optional _[_CustomPayloadTranslator]) – If you want to define your own serialization method for the payload, you can implement your functions for translation.
• output_translator (Optional _[_CustomPayloadTranslator]) – If you want to define your own serialization method for the output, you can implement your functions for translation.

class sagemaker.serve.CustomPayloadTranslator(content_type='application/custom', accept_type='application/custom')

Abstract base class for handling custom payload serialization and deserialization.

Provides a skeleton for customization requiring the overriding of theserialize_payload and deserialize_payload methods.

Parameters:

• content_type (str) – The content type of the endpoint input data.
• accept_type (str) – The content type of the data accepted from the endpoint.