Sync Benchmark Sample — OpenVINO™ documentation (original) (raw)

This sample demonstrates how to estimate performance of a model using Synchronous Inference Request API. It makes sense to use synchronous inference only in latency oriented scenarios. Models with static input shapes are supported. This sample does not have other configurable command-line arguments. Feel free to modify sample’s source code to try out different options. Before using the sample, refer to the following requirements:

The sample accepts any file format supported by core.read_model.
The sample has been validated with: the yolo-v3-tf and face-detection-0200 models.
To build the sample, use instructions available at Build the Sample Applicationssection in “Get Started with Samples” guide.

How It Works#

The sample compiles a model for a given device, randomly generates input data, performs synchronous inference multiple times for a given number of seconds. Then, it processes and reports performance results.

You can see the explicit description of each sample step at Integration Stepssection of “Integrate OpenVINO™ Runtime with Your Application” guide.

Running#

Python

python sync_benchmark.py (default: CPU)

C++

sync_benchmark (default: CPU)

To run the sample, you need to specify a model. You can get a model specific for your inference task from one of model repositories, such as TensorFlow Zoo, HuggingFace, or TensorFlow Hub.

Example#

Download a pre-trained model.
You can convert it by using:
Python
import openvino as ov
ov_model = ov.convert_model('./models/googlenet-v1')

or, when model is a Python model object

ov_model = ov.convert_model(googlenet-v1)
CLI
ovc ./models/googlenet-v1 3. Perform benchmarking, using the googlenet-v1 model on a CPU:
Python
python sync_benchmark.py googlenet-v1.xml
C++
sync_benchmark googlenet-v1.xml

Sample Output#

Python

The application outputs performance results.

[ INFO ] OpenVINO: [ INFO ] Build ................................. [ INFO ] Count: 2333 iterations [ INFO ] Duration: 10003.59 ms [ INFO ] Latency: [ INFO ] Median: 3.90 ms [ INFO ] Average: 4.29 ms [ INFO ] Min: 3.30 ms [ INFO ] Max: 10.11 ms [ INFO ] Throughput: 233.22 FPS

C++