Convert generative PyTorch models (original) (raw)

The AI Edge Torch Generative APIis a mobile-optimized library for converting transformer-based PyTorch models to the LiteRT format, enabling you to convert Large Language Models (LLMs) for on-device image and text generation with LiteRT and MediaPipe. The AI Edge Torch Generative API is currently CPU-only, with planned GPU and NPU support.

Mobile app developers can use the Edge Generative API to integrate PyTorch LLMs with the MediaPipe LLM Inference API to create apps that run those converted PyTorch models completely on-device.

For more information, see the Generative Torch API GitHub repo.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2024-08-30 UTC.