Packaging for deployment (original) (raw)
Toggle table of contents sidebar
BentoML provides a standardized format called Bentos for packaging AI/ML services. A Bento includes all the components required to run AI services, such as source code, Python dependencies, model artifacts, and configurations. This ensures your AI services are consistent and reproducible across different environments.
Define the runtime environment¶
Before building a Bento, you need to define the runtime environment of it. Here’s an example:
service.py¶
import bentoml
my_image = bentoml.images.Image(python_version="3.11")
.python_packages("torch", "transformers")
@bentoml.service( image=my_image, envs=[ {"name": "HF_TOKEN"}, # You can omit value and set it when deploying the Service {"name": "DB_HOST", "value": "localhost"} ] ) class Summarization: ...
Key environment fields:
python_version: Specifies the Python version to use. It defaults to the Python version in your build environment.python_packages: Lists the required Python packages. Alternatively, reference a requirements file using.requirements_file("requirements.txt").
In the @bentoml.service decorator, apply the runtime environment to your Service via image. Optionally, use the envs parameter to specify required environment variables.
See more available fields to customize your build.
Build a Bento¶
Run the following command in the same directory as your service.py file.
Note
By default, this command packages all files under the directory from which it is executed. To exclude specific files or directories, define them in a .bentoignore file.
After building, each Bento is automatically assigned a unique version. You can list all available Bentos using:
The bentoml build command is part of the bentoml deploy workflow. You should use this command only if you want to build a Bento without deploying it to BentoCloud. To deploy your project to BentoCloud directly, use bentoml deploy. For details, see Cloud deployment.
Containerize a Bento¶
To containerize a Bento with Docker, simply run bentoml containerize <bento_tag>. For example:
bentoml containerize summarization:latest
Note
For Mac computers with Apple silicon, you can specify the --platform option to avoid potential compatibility issues with some Python libraries.
bentoml containerize --platform=linux/amd64 summarization:latest
The Docker image’s tag is the same as the Bento tag by default. View the created Docker image:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE summarization lkpxx2u5o24wpxjr 79a06b402644 2 minutes ago 6.66GB
Run the Docker image locally:
docker run -it --rm -p 3000:3000 summarization:lkpxx2u5o24wpxjr serve
With the Docker image, you can run the model in any Docker-compatible environment.