Layout API Overview — OpenVINO™ documentation (original) (raw)

The concept of layout helps you (and your application) to understand what each particular dimension of input/output tensor means. For example, if your input has the {1, 3, 720, 1280} shape and the NCHW layout, it is clear that N(batch) = 1, C(channels) = 3, H(height) = 720, and W(width) = 1280. Without the layout information, the {1, 3, 720, 1280} tuple does not give any idea to your application on what these numbers mean and how to resize the input image to fit the expectations of the model.

With the NCHW layout, it is easier to understand what the {8, 3, 224, 224} model shape means. Without the layout, it is just a 4-dimensional tensor.

Below is a list of cases where input/output layout is important:

• Performing model modification:
  • Applying the preprocessing steps, such as subtracting means, dividing by scales, resizing an image, and converting RGB <-> BGR.
  • Setting/getting a batch for a model.
• Doing the same operations as used during the model conversion phase. For more information, refer to the:
  • Convert to OpenVINO
  • OpenVINO Model Conversion Tutorial
• Improving the readability of a model input and output.

Syntax of Layout#

Short Syntax#

The easiest way is to fully specify each dimension with one alphabet letter.

Python

from openvino import Layout layout = Layout('NCHW')

C++

layout = ov::Layout("NHWC");

This assigns N to the first dimension, C to the second, H to the third, and W to the fourth.

Advanced Syntax#

The advanced syntax allows assigning a word to a dimension. To do this, wrap a layout with square brackets [] and specify each name separated by a comma ,.

Python

Each dimension has name separated by comma

Layout is wrapped with square brackets

layout = Layout('[time,temperature,humidity]')

C++

// Each dimension has name separated by comma, layout is wrapped with square brackets layout = ov::Layout("[time,temperature,humidity]");

Partially Defined Layout#

If a certain dimension is not important, its name can be set to ?.

Python

First dimension is batch, 4th is 'channels'.

Others are not important for us

layout = Layout('N??C')

Or the same using advanced syntax

layout = Layout('[n,?,?,c]')

C++

// First dimension is batch, 4th is 'channels'. Others are not important for us layout = ov::Layout("N??C"); // Or the same using advanced syntax layout = ov::Layout("[n,?,?,c]");

Dynamic Layout#

If several dimensions are not important, an ellipsis ... can be used to specify those dimensions.

Python

First dimension is 'batch' others are whatever

layout = Layout('N...')

Second dimension is 'channels' others are whatever

layout = Layout('?C...')

Last dimension is 'channels' others are whatever

layout = Layout('...C')

C++

// First dimension is 'batch' others are whatever layout = ov::Layout("N...");

// Second dimension is 'channels' others are whatever layout = ov::Layout("?C...");

// Last dimension is 'channels' others are whatever layout = ov::Layout("...C");

Predefined Names#

A layout has some pre-defined dimension names, widely used in computer vision:

• N/Batch - batch size
• C/Channels - channels
• D/Depth - depth
• H/Height - height
• W/Width - width

These names are used in PreProcessing API. There is a set of helper functions to get appropriate dimension index from a layout.

Python

from openvino.runtime import layout_helpers

returns 0 for batch

layout_helpers.batch_idx(Layout('NCDHW'))

returns 1 for channels

layout_helpers.channels_idx(Layout('NCDHW'))

returns 2 for depth

layout_helpers.depth_idx(Layout('NCDHW'))

returns -2 for height

layout_helpers.height_idx(Layout('...HW'))

returns -1 for width

layout_helpers.width_idx(Layout('...HW'))

C++

// returns 0 for batch ov::layout::batch_idx("NCDHW");

// returns 1 for channels ov::layout::channels_idx("NCDHW");

// returns 2 for depth ov::layout::depth_idx("NCDHW");

// returns -2 for height ov::layout::height_idx("...HW");

// returns -1 for width ov::layout::width_idx("...HW");

Equality#

Layout names are case-insensitive, which means that Layout("NCHW") = Layout("nChW") = Layout("[N,c,H,w]").

Dump Layout#

A layout can be converted to a string in the advanced syntax format. It can be useful for debugging and serialization purposes.

Python

layout = Layout('NCHW') print(layout) # prints [N,C,H,W]

C++

layout = ov::Layout("NCHW"); std::cout << layout.to_string(); // prints [N,C,H,W]

Get layout from Model Input/Output#

OpenVINO provides helpers which provide a simple interface to get layout from Model input or output.

Python

Get layout for model input

layout = layout_helpers.get_layout(model.input("input_tensor_name"))

Get layout for model with single output

layout = layout_helpers.get_layout(model.output())

C++

// Get layout for model input layout = ov::layout::get_layout(model->input("input_tensor_name")); // Get layout for model with single output layout = ov::layout::get_layout(model->output());

See also#

• API Reference: ov::Layout C++ class