Supported Models | OpenVINO GenAI (original) (raw)
Models Compatibility
Other models with similar architectures may also work successfully even if not explicitly validated. Consider testing any unlisted models to verify compatibility with your specific use case.
Large Language Models (LLMs)
LoRA Support
LLM pipeline supports LoRA adapters.
info
The LLM pipeline can work with other similar topologies produced by optimum-intel with the same model signature. The model is required to have the following inputs after the conversion:
input_idscontains the tokens.attention_maskis filled with1.beam_idxselects beams.position_ids(optional) encodes a position of currently generating token in the sequence and a singlelogitsoutput.
note
Models should belong to the same family and have the same tokenizers.
Image Generation Models
Visual Language Models (VLMs)
LoRA Support
VLM pipeline does not support LoRA adapters.
VLM Models Notes
InternVL2
To convert InternVL2 models, timm and einops are required:
pip install timm einops
MiniCPMO
openbmb/MiniCPM-o-2_6doesn't supporttransformers>=4.52which is required foroptimum-cliexport.--task image-text-to-textis required foroptimum-cli export openvino --trust-remote-codebecauseimage-text-to-textisn'tMiniCPM-o-2_6's native task.
phi3_v
Models' configs aren't consistent. It's required to override the default eos_token_id with the one from a tokenizer:
generation_config.set_eos_token_id(pipe.get_tokenizer().get_eos_token_id())
phi4mm
Apply https://huggingface.co/microsoft/Phi-4-multimodal-instruct/discussions/78/files to fix the model export for transformers>=4.50
Speech Recognition Models (Whisper-based)
LoRA Support
Speech recognition pipeline does not support LoRA adapters.
| Architecture | Models | Example HuggingFace Models |
|---|---|---|
| WhisperForConditionalGeneration | Whisper | openai/whisper-tinyopenai/whisper-tiny.enopenai/whisper-baseopenai/whisper-base.enopenai/whisper-smallopenai/whisper-small.enopenai/whisper-mediumopenai/whisper-medium.enopenai/whisper-large-v3 |
| Distil-Whisper | distil-whisper/distil-small.endistil-whisper/distil-medium.endistil-whisper/distil-large-v3 |
Speech Generation Models
LoRA Support
Speech generation pipeline does not support LoRA adapters.
| Architecture | Models | Example HuggingFace Models |
|---|---|---|
| SpeechT5ForTextToSpeech | SpeechT5 TTS | microsoft/speecht5_tts |
Text Embeddings Models
LoRA Support
Text embeddings pipeline does not support LoRA adapters.
Text Embeddings Models Notes
Qwen3 Embedding models require --task feature-extraction during the conversion with optimum-cli.
Text Rerank Models
LoRA Support
Text rerank pipeline does not support LoRA adapters.
| Architecture | `optimum-cli` task | Example HuggingFace Models |
|---|---|---|
| BertForSequenceClassification | text-classification | cross-encoder/ms-marco-MiniLM-L2-v2cross-encoder/ms-marco-MiniLM-L4-v2cross-encoder/ms-marco-MiniLM-L6-v2cross-encoder/ms-marco-MiniLM-L12-v2cross-encoder/ms-marco-TinyBERT-L2-v2tomaarsen/reranker-MiniLM-L12-gooaq-bce |
| XLMRobertaForSequenceClassification | text-classification | BAAI/bge-reranker-v2-m3 |
| ModernBertForSequenceClassification | text-classification | tomaarsen/reranker-ModernBERT-base-gooaq-bcetomaarsen/reranker-ModernBERT-large-gooaq-bceAlibaba-NLP/gte-reranker-modernbert-base |
| Qwen3ForCausalLM | text-generation-with-past | Qwen/Qwen3-Reranker-0.6BQwen/Qwen3-Reranker-4BQwen/Qwen3-Reranker-8B |
Text Rerank Models Notes
Text Rerank models require appropriate --task provided during the conversion with optimum-cli. Task can be found in the table above.
Hugging Face Notes
Some models may require access request submission on the Hugging Face page to be downloaded.
If https://huggingface.co/ is down, the conversion step won't be able to download the models.