Overview (original) (raw)

Back to top

View this page

Edit this page

Toggle table of contents sidebar

This section provides the tutorials for a curated list of example projects to help you learn how BentoML can be used for different scenarios. See the following lists for a complete collection of BentoML example projects. Browse through different categories to find the example that best suits your needs.

LLMs¶

Deploy an OpenAI-compatible LLM API service with BentoML and vLLM:

• DeepSeek R1 Distill of Llama 3.3 70B
• Llama 3.2 11B Vision
• Pixtral 12B
• Mistral Small 24B
• Check out the BentoVLLM project to see more supported models.

Customize your LLM inference runtime:

• vLLM
• TensorRT-LLM
• LMDeploy
• MLC-LLM
• SGLang
• Hugging Face TGI

Compound AI systems¶

Build and scale compound AI systems with BentoML:

• Agent: Function calling
• Agent: LangGraph
• Multi-agent: CrewAI
• LLM safety: ShieldGemma
• RAG: LlamaIndex
• Phone call agent
• Multi-LLM routing

Image and video¶

Serve text-to-image and image-to-image models with BentoML:

• ComfyUI workflows as APIs
• Stable Diffusion 3.5 Large Turbo
• Stable Diffusion 3 Medium
• Stable Diffusion XL Turbo
• ControlNet
• Check out the BentoDiffusion project to see more examples

Audio¶

Serve text-to-speech and speech-to-text models with BentoML:

• ChatTTS
• XTTS
• XTTS with a streaming endpoint
• WhisperX
• Bark
• Moshi

Computer vision¶

Serve computer vision models with BentoML:

• YOLO: Object detection
• ResNet: Image classification
• EasyOCR: Optical character recognition

Embeddings¶

Build embedding inference APIs with BentoML:

• SentenceTransformers
• CLIP
• ColPali

Custom models¶

Serve custom models with BentoML:

• MLflow
• XGBoost

Others¶

• BLIP inference API for image captioning and VQA (Visual Question Answering)
• Serving Moirai: Time-series forecasting as a service