Install OpenVINO GenAI | OpenVINO GenAI (original) (raw)

OpenVINO GenAI is available for installation via PyPI, Archiveand npm distributions and supports Linux, Windows and macOS platforms.

pip install openvino-genai

Refer to the Install Guide for detailed instructions.

To build OpenVINO GenAI library from source, refer to the Build Instructions.

info

Please make sure that you are following the versions compatibility rules, refer to the OpenVINO GenAI Dependencies for more information.

System Requirements​

OpenVINO GenAI is built on top of OpenVINO Runtime and shares the same system requirements.

Refer to the OpenVINO System Requirements for more details.

OpenVINO GenAI Dependencies​

OpenVINO GenAI depends on OpenVINO and OpenVINO Tokenizers.

When installing OpenVINO GenAI from PyPi, the same versions of OpenVINO and OpenVINO Tokenizers are used (e.g. openvino==2024.3.0 and openvino-tokenizers==2024.3.0.0 are installed for openvino-genai==2024.3.0). If you update one of the dependency packages (e.g. pip install openvino --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly), versions might be incompatible due to different ABI and running OpenVINO GenAI can result in errors (e.g. ImportError: libopenvino.so.2430: cannot open shared object file: No such file or directory). Having packages version in format <MAJOR>.<MINOR>.<PATCH>.<REVISION>, only <REVISION> part of the full version can be varied to ensure ABI compatibility, while changing <MAJOR>, <MINOR> or <PATCH> parts of the version might break ABI.

GenAI, Tokenizers, and OpenVINO wheels for Linux on PyPI are compiled with _GLIBCXX_USE_CXX11_ABI=0 to cover a wider range of platforms. In contrast, C++ archive distributions for Ubuntu are compiled with _GLIBCXX_USE_CXX11_ABI=1. It is not possible to mix different Application Binary Interfaces (ABIs) because doing so results in a link error. This incompatibility prevents the use of, for example, OpenVINO from C++ archive distributions alongside GenAI from PyPI.

If you want to try OpenVINO GenAI with different dependencies versions (not prebuilt packages as archives or python wheels), build OpenVINO GenAI library from source.