Transition from Legacy Conversion API — OpenVINO™ documentation (original) (raw)

In the 2023.1 OpenVINO release OpenVINO Model Converter was introduced with the corresponding Python API: openvino.convert_model method. ovc and openvino.convert_model represent a lightweight alternative of mo and openvino.tools.mo.convert_model which are considered legacy API now. In this article, all the differences between mo and ovc are summarized and the transition guide from the legacy API to the new API is provided.

Parameters Comparison#

The comparison of parameters between ov.convert_model() / OVC and mo.convert_model() / MO.

Transition from Legacy API to New API#

mo.convert_model() provides a wide range of preprocessing parameters. Most of these parameters have analogs in OVC or can be replaced with functionality from ov.PrePostProcessor class. Here is the guide to transition from legacy model preprocessing to new API preprocessing.

input_shape#

Python

CLI

batch#

Python

CLI

mean_values#

Python

CLI

scale_values#

Python

CLI

reverse_input_channels#

Python

CLI

source_layout#

Python

CLI

target_layout#

Python

CLI

layout#

Python

CLI

transform#

Python

CLI

Cutting Off Parts of a Model#

Performing surgery by cutting model inputs and outputs from a model is no longer available in the new conversion API. Instead, we recommend performing the cut in the original framework. Below are examples of model cutting of TensorFlow protobuf, TensorFlow SavedModel, and ONNX formats with the legacy conversion API, compared to achieving the same cut with tools provided by the Tensorflow and ONNX frameworks. For PyTorch, TensorFlow 2 Keras, and PaddlePaddle, we recommend changing the original model code to perform the model cut.

Note: This guide does not cover the cutting a model by input port of an operation that MO tool provides using input and output options, for example, –input 1:name_op.

PyTorch#

Model cut for PyTorch is not available in legacy API.

When it is needed to remove a whole module from the model it is possible to replace such modules with Identity. Below is the example of removing conv1 and bn1 modules at the input and fc module at the output of the resnet50 model.

import openvino as ov import torch import torchvision from torch.nn import Identity

Load pretrained model

model = torchvision.models.resnet50(weights='DEFAULT')

input cut

model.conv1 = Identity() model.bn1 = Identity()

output cut

model.fc = Identity()

convert and compile the model

ov_model = ov.convert_model(model, input=([-1,64,-1,-1], torch.float32)) compiled_model = ov.compile_model(ov_model)

When it is needed to remove one or more outputs from the model it is possible to create a wrapper for the model and only output the needed output. Below is the example of removing second output from the model.

import openvino as ov import torch

Example of model with multiple outputs

class Model(torch.nn.Module): def init(self): super(Model, self).init() self.linear1 = torch.nn.Linear(100, 200) self.activation1 = torch.nn.ReLU() self.linear2 = torch.nn.Linear(200, 10) self.activation2 = torch.nn.Sigmoid()

def forward(self, x):
    x = self.linear1(x)
    x = self.activation1(x)
    y = self.linear2(x)
    y = self.activation2(y)
    return x, y

New model, where some outputs are cut

class CutModel(torch.nn.Module): def init(self): super(CutModel, self).init() self.model = Model()

def forward(self, x):

    # get first output
    x, _ = self.model(x)

    return x

Model with output cut

cut_model = CutModel()

convert and compile the model

ov_model = ov.convert_model(cut_model, input=([-1,-1,-1], torch.float32)) compiled_model = ov.compile_model(ov_model)

TensorFlow protobuf format / tf.Graph / tf.GraphDef#

Legacy API.

import openvino as ov import openvino.tools.mo as mo

import tensorflow as tf

def load_graph(model_path): graph_def = tf.compat.v1.GraphDef() with open(model_path, "rb") as f: graph_def.ParseFromString(f.read()) with tf.compat.v1.Graph().as_default() as graph: tf.graph_util.import_graph_def(graph_def, name="") return graph

Load TF model

graph = load_graph("/path_to_model/HugeCTR.pb")

Convert the model with input and output cut

input_name = "concat" output_name = "MatVec_3/Squeeze" ov_model = mo.convert_model(graph, input=(input_name, [-1, -1]), output=output_name)

Compile the model

compiled_model = ov.compile_model(ov_model)

Model cut in original FW.

import openvino as ov import tensorflow as tf

from tensorflow.python.tools.strip_unused_lib import strip_unused

def load_graph(model_path): graph_def = tf.compat.v1.GraphDef() with open(model_path, "rb") as f: graph_def.ParseFromString(f.read()) with tf.compat.v1.Graph().as_default() as graph: tf.graph_util.import_graph_def(graph_def, name="") return graph

Load TF model

graph = load_graph("/path_to_model/HugeCTR.pb")

Cut the model

input_name = "concat" output_name = "MatVec_3/Squeeze" graph_def = graph.as_graph_def() new_graph_def = strip_unused(graph_def, [input_name], [output_name], tf.float32.as_datatype_enum)

Convert and compile model

ov_model = ov.convert_model(new_graph_def, input=[-1, -1]) cmp_model = ov.compile_model(ov_model)

TensorFlow SavedModel format#

Model cut for SavedModel format is not available in legacy API.

Example of model cut in original FW.

import openvino as ov import tensorflow_hub as hub

import tensorflow as tf from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2 from tensorflow.python.tools.strip_unused_lib import strip_unused

Load TF model

model = hub.load("https://tfhub.dev/svampeatlas/vision/embedder/fungi_V2/1?tf-hub-format=compressed")

Convert model to GraphDef

model_func = model.signatures["default"] frozen_func = convert_variables_to_constants_v2(model_func) graph_def = frozen_func.graph.as_graph_def()

Cut the model

input_name = 'InceptionV4/InceptionV4/Conv2d_2b_3x3/Relu' output_name = 'InceptionV4/InceptionV4/Mixed_7c/concat' new_graph_def = strip_unused(graph_def, [input_name], [output_name], tf.float32.as_datatype_enum)

Convert and compile the model

ov_model = ov.convert_model(new_graph_def) compiled_model = ov.compile_model(ov_model)

ONNX#

Legacy API.

import openvino as ov import openvino.tools.mo as mo

input_path = "/path_to_model/yolov8x.onnx"

Convert model and perform input and output cut

input_name = "/model.2/Concat_output_0" output_name = "/model.22/Concat_3_output_0" ov_model = mo.convert_model(input_path, input=input_name, output=output_name)

Compile model

ov.compile_model(ov_model)

Model cut in original FW.

import onnx import openvino as ov

input_path = "/path_to_model/yolov8x.onnx"

Cut the model

input_name = "/model.2/Concat_output_0" output_name = "/model.22/Concat_3_output_0" cut_model_path = "/path_to_model/yolov8x_cut.onnx" onnx.utils.extract_model(input_path, cut_model_path, [input_name], [output_name])

Convert model

ov_model = ov.convert_model(cut_model_path)

Compile model

ov.compile_model(ov_model)

Supported Frameworks in MO vs OVC#

ov.convert_model() and OVC tool support conversion from PyTorch, TF, TF Lite, ONNX, PaddlePaddle.