Deploy Python Lambda functions with container images (original) (raw)

There are three ways to build a container image for a Python Lambda function:

Using an AWS base image for Python
The AWS base images are preloaded with a language runtime, a runtime interface client to manage the interaction between Lambda and your function code, and a runtime interface emulator for local testing.
Using an AWS OS-only base image
AWS OS-only base images contain an Amazon Linux distribution and the runtime interface emulator. These images are commonly used to create container images for compiled languages, such as Go and Rust, and for a language or language version that Lambda doesn't provide a base image for, such as Node.js 19. You can also use OS-only base images to implement a custom runtime. To make the image compatible with Lambda, you must include the runtime interface client for Python in the image.
Using a non-AWS base image
You can use an alternative base image from another container registry, such as Alpine Linux or Debian. You can also use a custom image created by your organization. To make the image compatible with Lambda, you must include the runtime interface client for Python in the image.

This page explains how to build, test, and deploy container images for Lambda.

Topics

AWS base images for Python
Using an AWS base image for Python
Using an alternative base image with the runtime interface client

AWS base images for Python

AWS provides the following base images for Python:

Amazon ECR repository: gallery.ecr.aws/lambda/python

Python 3.12 and later base images are based on the Amazon Linux 2023 minimal container image. The Python 3.8-3.11 base images are based on the Amazon Linux 2 image. AL2023-based images provide several advantages over Amazon Linux 2, including a smaller deployment footprint and updated versions of libraries such as glibc.

AL2023-based images use microdnf (symlinked as dnf) as the package manager instead of yum, which is the default package manager in Amazon Linux 2. microdnf is a standalone implementation of dnf. For a list of packages that are included in AL2023-based images, refer to the Minimal Container columns in Comparing packages installed on Amazon Linux 2023 Container Images. For more information about the differences between AL2023 and Amazon Linux 2, see Introducing the Amazon Linux 2023 runtime for AWS Lambda on the AWS Compute Blog.

Note

To run AL2023-based images locally, including with AWS Serverless Application Model (AWS SAM), you must use Docker version 20.10.10 or later.

Dependency search path in the base images

When you use an import statement in your code, the Python runtime searches the directories in its search path until it finds the module or package. By default, the runtime searches the {LAMBDA_TASK_ROOT} directory first. If you include a version of a runtime-included library in your image, your version will take precedence over the version that's included in the runtime.

Other steps in the search path depend on which version of the Lambda base image for Python you're using:

Python 3.11 and later: Runtime-included libraries and pip-installed libraries are installed in the /var/lang/lib/python3.11/site-packages directory. This directory has precedence over /var/runtime in the search path. You can override the SDK by using pip to install a newer version. You can use pip to verify that the runtime-included SDK and its dependencies are compatible with any packages that you install.
Python 3.8-3.10: Runtime-included libraries are installed in the /var/runtime directory. Pip-installed libraries are installed in the /var/lang/lib/python3.x/site-packages directory. The /var/runtime directory has precedence over /var/lang/lib/python3.x/site-packages in the search path.

You can see the full search path for your Lambda function by adding the following code snippet.

import sys
      
search_path = sys.path
print(search_path)

Using an AWS base image for Python

To complete the steps in this section, you must have the following:

AWS CLI version 2
Docker (minimum version 25.0.0)
The Docker buildx plugin.
Python

To create a container image from an AWS base image for Python

Create a directory for the project, and then switch to that directory.

mkdir example  
cd example

Create a new file called lambda_function.py. You can add the following sample function code to the file for testing, or use your own.

Example Python function

import sys  
def handler(event, context):  
    return 'Hello from AWS Lambda using Python' + sys.version + '!'

Create a new file called requirements.txt. If you're using the sample function code from the previous step, you can leave the file empty because there are no dependencies. Otherwise, list each required library. For example, here's what your requirements.txt should look like if your function uses the AWS SDK for Python (Boto3):

Example requirements.txt

Create a new Dockerfile with the following configuration:
- Set the FROM property to the URI of the base image.
- Use the COPY command to copy the function code and runtime dependencies to {LAMBDA_TASK_ROOT}, a Lambda-defined environment variable.
- Set the CMD argument to the Lambda function handler.
  Note that the example Dockerfile does not include a USER instruction. When you deploy a container image to Lambda, Lambda automatically defines a default Linux user with least-privileged permissions. This is different from standard Docker behavior which defaults to the root user when no USER instruction is provided.

Example Dockerfile

FROM public.ecr.aws/lambda/python:3.12  
# Copy requirements.txt  
COPY requirements.txt ${LAMBDA_TASK_ROOT}  
# Install the specified packages  
RUN pip install -r requirements.txt  
# Copy function code  
COPY lambda_function.py ${LAMBDA_TASK_ROOT}  
# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)  
CMD [ "lambda_function.handler" ]

Build the Docker image with the docker build command. The following example names the image docker-image and gives it the test tag. To make your image compatible with Lambda, you must use the --provenance=false option.

docker buildx build --platform linux/amd64 --provenance=false -t docker-image:test .

Note

The command specifies the --platform linux/amd64 option to ensure that your container is compatible with the Lambda execution environment regardless of the architecture of your build machine. If you intend to create a Lambda function using the ARM64 instruction set architecture, be sure to change the command to use the --platform linux/arm64 option instead.

Start the Docker image with the docker run command. In this example,docker-image is the image name and test is the tag.

docker run --platform linux/amd64 -p 9000:8080 docker-image:test

This command runs the image as a container and creates a local endpoint atlocalhost:9000/2015-03-31/functions/function/invocations.

Note

If you built the Docker image for the ARM64 instruction set architecture, be sure to use the --platform linux/`arm64` option instead of --platform linux/`amd64`. 2. From a new terminal window, post an event to the local endpoint.
Linux/macOS
In Linux and macOS, run the following curl command:

curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

This command invokes the function with an empty event and returns a response. If you're using your own function code rather than the sample function code, you might want to invoke the function with a JSON payload. Example:

curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"payload":"hello world!"}'

PowerShell
In PowerShell, run the following Invoke-WebRequest command:

Invoke-WebRequest -Uri "http://localhost:9000/2015-03-31/functions/function/invocations" -Method Post -Body '{}' -ContentType "application/json"

Invoke-WebRequest -Uri "http://localhost:9000/2015-03-31/functions/function/invocations" -Method Post -Body '{"payload":"hello world!"}' -ContentType "application/json"

Get the container ID.

docker ps

Use the docker kill command to stop the container. In this command, replace 3766c4ab331c with the container ID from the previous step.

docker kill 3766c4ab331c

To upload the image to Amazon ECR and create the Lambda function

Run the get-login-password command to authenticate the Docker CLI to your Amazon ECR registry.
- Set the --region value to the AWS Region where you want to create the Amazon ECR repository.
- Replace 111122223333 with your AWS account ID.

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 111122223333.dkr.ecr.us-east-1.amazonaws.com

Create a repository in Amazon ECR using the create-repository command.

aws ecr create-repository --repository-name hello-world --region us-east-1 --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE

Note

The Amazon ECR repository must be in the same AWS Region as the Lambda function.
If successful, you see a response like this:

{  
    "repository": {  
        "repositoryArn": "arn:aws:ecr:us-east-1:111122223333:repository/hello-world",  
        "registryId": "111122223333",  
        "repositoryName": "hello-world",  
        "repositoryUri": "111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world",  
        "createdAt": "2023-03-09T10:39:01+00:00",  
        "imageTagMutability": "MUTABLE",  
        "imageScanningConfiguration": {  
            "scanOnPush": true  
        },  
        "encryptionConfiguration": {  
            "encryptionType": "AES256"  
        }  
    }  
}

Copy the repositoryUri from the output in the previous step.
Run the docker tag command to tag your local image into your Amazon ECR repository as the latest version. In this command:
- docker-image:test is the name and tag of your Docker image. This is the image name and tag that you specified in the docker build command.
- Replace <ECRrepositoryUri> with the repositoryUri that you copied. Make sure to include :latest at the end of the URI.

docker tag docker-image:test <ECRrepositoryUri>:latest

Example:

docker tag docker-image:test 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

Run the docker push command to deploy your local image to the Amazon ECR repository. Make sure to include :latest at the end of the repository URI.

docker push 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

Create an execution role for the function, if you don't already have one. You need the Amazon Resource Name (ARN) of the role in the next step.
Create the Lambda function. For ImageUri, specify the repository URI from earlier. Make sure to include :latest at the end of the URI.

aws lambda create-function \  
  --function-name hello-world \  
  --package-type Image \  
  --code ImageUri=111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest \  
  --role arn:aws:iam::111122223333:role/lambda-ex

Note

You can create a function using an image in a different AWS account, as long as the image is in the same Region as the Lambda function. For more information, see Amazon ECR cross-account permissions. 8. Invoke the function.

aws lambda invoke --function-name hello-world response.json

You should see a response like this:

{  
  "ExecutedVersion": "$LATEST",  
  "StatusCode": 200  
}

To see the output of the function, check the response.json file.

To update the function code, you must build the image again, upload the new image to the Amazon ECR repository, and then use the update-function-code command to deploy the image to the Lambda function.

Lambda resolves the image tag to a specific image digest. This means that if you point the image tag that was used to deploy the function to a new image in Amazon ECR, Lambda doesn't automatically update the function to use the new image.

To deploy the new image to the same Lambda function, you must use the update-function-code command, even if the image tag in Amazon ECR remains the same. In the following example, the --publish option creates a new version of the function using the updated container image.

aws lambda update-function-code \
  --function-name hello-world \
  --image-uri 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest \
  --publish

Using an alternative base image with the runtime interface client

If you use an OS-only base image or an alternative base image, you must include the runtime interface client in your image. The runtime interface client extends the Runtime API, which manages the interaction between Lambda and your function code.

Install the the runtime interface client for Python using the pip package manager:

pip install awslambdaric

You can also download the Python runtime interface client from GitHub.

The following example demonstrates how to build a container image for Python using a non-AWS base image. The example Dockerfile uses an official Python base image. The Dockerfile includes the runtime interface client for Python.

To complete the steps in this section, you must have the following:

AWS CLI version 2
Docker (minimum version 25.0.0)
The Docker buildx plugin.
Python

To create a container image from a non-AWS base image

Create a directory for the project, and then switch to that directory.

mkdir example  
cd example

Create a new file called lambda_function.py. You can add the following sample function code to the file for testing, or use your own.

Example Python function

import sys  
def handler(event, context):  
    return 'Hello from AWS Lambda using Python' + sys.version + '!'

Create a new file called requirements.txt. If you're using the sample function code from the previous step, you can leave the file empty because there are no dependencies. Otherwise, list each required library. For example, here's what your requirements.txt should look like if your function uses the AWS SDK for Python (Boto3):

Example requirements.txt

Create a new Dockerfile. The following Dockerfile uses an official Python base image instead of an AWS base image. The Dockerfile includes the runtime interface client, which makes the image compatible with Lambda. The following example Dockerfile uses a multi-stage build.
- Set the FROM property to the base image.
- Set the ENTRYPOINT to the module that you want the Docker container to run when it starts. In this case, the module is the runtime interface client.
- Set the CMD to the Lambda function handler.
  Note that the example Dockerfile does not include a USER instruction. When you deploy a container image to Lambda, Lambda automatically defines a default Linux user with least-privileged permissions. This is different from standard Docker behavior which defaults to the root user when no USER instruction is provided.

Example Dockerfile

# Define custom function directory  
ARG FUNCTION_DIR="/function"  
FROM python:3.12 AS build-image  
# Include global arg in this stage of the build  
ARG FUNCTION_DIR  
# Copy function code  
RUN mkdir -p ${FUNCTION_DIR}  
COPY . ${FUNCTION_DIR}  
# Install the function's dependencies  
RUN pip install \  
    --target ${FUNCTION_DIR} \  
        awslambdaric  
# Use a slim version of the base Python image to reduce the final image size  
FROM python:3.12-slim  
# Include global arg in this stage of the build  
ARG FUNCTION_DIR  
# Set working directory to function root directory  
WORKDIR ${FUNCTION_DIR}  
# Copy in the built dependencies  
COPY --from=build-image <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>F</mi><mi>U</mi><mi>N</mi><mi>C</mi><mi>T</mi><mi>I</mi><mi>O</mi><msub><mi>N</mi><mi>D</mi></msub><mi>I</mi><mi>R</mi></mrow><annotation encoding="application/x-tex">{FUNCTION_DIR} </annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8333em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.13889em;">F</span><span class="mord mathnormal" style="margin-right:0.10903em;">U</span><span class="mord mathnormal" style="margin-right:0.13889em;">NCT</span><span class="mord mathnormal" style="margin-right:0.07847em;">I</span><span class="mord mathnormal" style="margin-right:0.02778em;">O</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.10903em;">N</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.109em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">D</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord mathnormal" style="margin-right:0.07847em;">I</span><span class="mord mathnormal" style="margin-right:0.00773em;">R</span></span></span></span></span>{FUNCTION_DIR}  
# Set runtime interface client as default command for the container runtime  
ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]  
# Pass the name of the function handler as an argument to the runtime  
CMD [ "lambda_function.handler" ]

Build the Docker image with the docker build command. The following example names the image docker-image and gives it the test tag. To make your image compatible with Lambda, you must use the --provenance=false option.

docker buildx build --platform linux/amd64 --provenance=false -t docker-image:test .

Note

Use the runtime interface emulator to locally test the image. You can build the emulator into your image or use the following procedure to install it on your local machine.

To install and run the runtime interface emulator on your local machine

From your project directory, run the following command to download the runtime interface emulator (x86-64 architecture) from GitHub and install it on your local machine.
Linux/macOS

mkdir -p ~/.aws-lambda-rie && \  
    curl -Lo ~/.aws-lambda-rie/aws-lambda-rie https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie && \  
    chmod +x ~/.aws-lambda-rie/aws-lambda-rie

To install the arm64 emulator, replace the GitHub repository URL in the previous command with the following:

https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie-arm64

PowerShell

if (-not (Test-Path $dirPath)) {  
    New-Item -Path $dirPath -ItemType Directory  
}  
        
$downloadLink = "https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie"   <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>d</mi><mi>e</mi><mi>s</mi><mi>t</mi><mi>i</mi><mi>n</mi><mi>a</mi><mi>t</mi><mi>i</mi><mi>o</mi><mi>n</mi><mi>P</mi><mi>a</mi><mi>t</mi><mi>h</mi><mo>=</mo><mi mathvariant="normal">&quot;</mi></mrow><annotation encoding="application/x-tex">destinationPath = &quot;</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal">d</span><span class="mord mathnormal">es</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ina</span><span class="mord mathnormal">t</span><span class="mord mathnormal">i</span><span class="mord mathnormal">o</span><span class="mord mathnormal">n</span><span class="mord mathnormal" style="margin-right:0.13889em;">P</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span><span class="mord mathnormal">h</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord">&quot;</span></span></span></span>HOME\.aws-lambda-rie\aws-lambda-rie"  
Invoke-WebRequest -Uri <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>d</mi><mi>o</mi><mi>w</mi><mi>n</mi><mi>l</mi><mi>o</mi><mi>a</mi><mi>d</mi><mi>L</mi><mi>i</mi><mi>n</mi><mi>k</mi><mo>−</mo><mi>O</mi><mi>u</mi><mi>t</mi><mi>F</mi><mi>i</mi><mi>l</mi><mi>e</mi></mrow><annotation encoding="application/x-tex">downloadLink -OutFile </annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.7778em;vertical-align:-0.0833em;"></span><span class="mord mathnormal">d</span><span class="mord mathnormal">o</span><span class="mord mathnormal" style="margin-right:0.02691em;">w</span><span class="mord mathnormal">n</span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mord mathnormal">o</span><span class="mord mathnormal">a</span><span class="mord mathnormal">d</span><span class="mord mathnormal">L</span><span class="mord mathnormal" style="margin-right:0.03148em;">ink</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">O</span><span class="mord mathnormal">u</span><span class="mord mathnormal" style="margin-right:0.13889em;">tF</span><span class="mord mathnormal">i</span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mord mathnormal">e</span></span></span></span>destinationPath

To install the arm64 emulator, replace the $downloadLink with the following:

https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie-arm64

Start the Docker image with the docker run command. Note the following:
- docker-image is the image name and test is the tag.
- /usr/local/bin/python -m awslambdaric lambda_function.handler is the ENTRYPOINT followed by the CMD from your Dockerfile.
  Linux/macOS

docker run --platform linux/amd64 -d -v ~/.aws-lambda-rie:/aws-lambda -p 9000:8080 \  
    --entrypoint /aws-lambda/aws-lambda-rie \  
    docker-image:test \  
        /usr/local/bin/python -m awslambdaric lambda_function.handler

PowerShell

docker run --platform linux/amd64 -d -v "$HOME\.aws-lambda-rie:/aws-lambda" -p 9000:8080 `  
--entrypoint /aws-lambda/aws-lambda-rie `  
docker-image:test `  
    /usr/local/bin/python -m awslambdaric lambda_function.handler

This command runs the image as a container and creates a local endpoint atlocalhost:9000/2015-03-31/functions/function/invocations.

Note

If you built the Docker image for the ARM64 instruction set architecture, be sure to use the --platform linux/`arm64` option instead of --platform linux/`amd64`. 3. Post an event to the local endpoint.
Linux/macOS
In Linux and macOS, run the following curl command:

curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"payload":"hello world!"}'

PowerShell
In PowerShell, run the following Invoke-WebRequest command:

Invoke-WebRequest -Uri "http://localhost:9000/2015-03-31/functions/function/invocations" -Method Post -Body '{}' -ContentType "application/json"

Invoke-WebRequest -Uri "http://localhost:9000/2015-03-31/functions/function/invocations" -Method Post -Body '{"payload":"hello world!"}' -ContentType "application/json"

Get the container ID.

docker ps

Use the docker kill command to stop the container. In this command, replace 3766c4ab331c with the container ID from the previous step.

docker kill 3766c4ab331c

To upload the image to Amazon ECR and create the Lambda function

Run the get-login-password command to authenticate the Docker CLI to your Amazon ECR registry.
- Set the --region value to the AWS Region where you want to create the Amazon ECR repository.
- Replace 111122223333 with your AWS account ID.

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 111122223333.dkr.ecr.us-east-1.amazonaws.com

Create a repository in Amazon ECR using the create-repository command.

aws ecr create-repository --repository-name hello-world --region us-east-1 --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE

Note

The Amazon ECR repository must be in the same AWS Region as the Lambda function.
If successful, you see a response like this:

{  
    "repository": {  
        "repositoryArn": "arn:aws:ecr:us-east-1:111122223333:repository/hello-world",  
        "registryId": "111122223333",  
        "repositoryName": "hello-world",  
        "repositoryUri": "111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world",  
        "createdAt": "2023-03-09T10:39:01+00:00",  
        "imageTagMutability": "MUTABLE",  
        "imageScanningConfiguration": {  
            "scanOnPush": true  
        },  
        "encryptionConfiguration": {  
            "encryptionType": "AES256"  
        }  
    }  
}

Copy the repositoryUri from the output in the previous step.
Run the docker tag command to tag your local image into your Amazon ECR repository as the latest version. In this command:
- docker-image:test is the name and tag of your Docker image. This is the image name and tag that you specified in the docker build command.
- Replace <ECRrepositoryUri> with the repositoryUri that you copied. Make sure to include :latest at the end of the URI.

docker tag docker-image:test <ECRrepositoryUri>:latest

Example:

docker tag docker-image:test 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

Run the docker push command to deploy your local image to the Amazon ECR repository. Make sure to include :latest at the end of the repository URI.

docker push 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest

Create an execution role for the function, if you don't already have one. You need the Amazon Resource Name (ARN) of the role in the next step.
Create the Lambda function. For ImageUri, specify the repository URI from earlier. Make sure to include :latest at the end of the URI.

aws lambda create-function \  
  --function-name hello-world \  
  --package-type Image \  
  --code ImageUri=111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest \  
  --role arn:aws:iam::111122223333:role/lambda-ex

Note

aws lambda invoke --function-name hello-world response.json

You should see a response like this:

{  
  "ExecutedVersion": "$LATEST",  
  "StatusCode": 200  
}

To see the output of the function, check the response.json file.

aws lambda update-function-code \
  --function-name hello-world \
  --image-uri 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest \
  --publish

For an example of how to create a Python image from an Alpine base image, see Container image support for Lambda on the AWS Blog.