Profile Inference Run - MATLAB & Simulink (original) (raw)

Main Content

This example shows how to retrieve the prediction and profiler results for the ResNet-18 network. View the network prediction and performance data for the layers, convolution module and fully connected modules in your pretrained deep learning network.

Create an object of class Workflow by using thedlhdl.Workflow class.
See, Create Workflow Object by using Property Name Value Pairs.
Set a pretrained deep learning network and bitstream for the workflow object.
See, Create Workflow Object by using Property Name Value Pairs.
Create an object of class dlhdl.Target and specify the target vendor and interface. See, dlhdl.Target.
To deploy the network on a specified target FPGA board, call thedeploy method for the workflow object. See, deploy.
Call the predict function for the workflow object. Provide an array of images as the InputImage parameter. Provide arguments to turn on the profiler. See Classify Images on FPGA Using Quantized Neural Network.
The labels classifying the images are stored in a structurestruct and displayed on the screen. The performance parameters of speed and latency are returned in a structure struct.

Use this image to run this code:

snet = resnet18; hT = dlhdl.Target('Xilinx','Interface','Ethernet'); hW = dlhdl.Workflow('Net',snet,'Bitstream','zcu102_single','Target',hT); hW.deploy; image = imread('zebra.jpeg'); inputImg = imresize(image, [224, 224]); imshow(inputImg); [prediction, speed] = hW.predict(single(inputImg),'Profile','on'); [val, idx] = max(prediction); snet.Layers(end).ClassNames{idx}

Finished writing input activations.

Running single input activations.

          Deep Learning Processor Profiler Performance Results

               LastFrameLatency(cycles)   LastFrameLatency(seconds)       FramesNum      Total Latency     Frames/s
                     -------------             -------------              ---------        ---------       ---------

Network 23659630 0.10754 1 23659630 9.3 conv1 2224115 0.01011 pool1 572867 0.00260 res2a_branch2a 972699 0.00442 res2a_branch2b 972568 0.00442 res2a 209312 0.00095 res2b_branch2a 972733 0.00442 res2b_branch2b 973022 0.00442 res2b 209736 0.00095 res3a_branch2a 747507 0.00340 res3a_branch2b 904291 0.00411 res3a_branch1 538763 0.00245 res3a 104750 0.00048 res3b_branch2a 904389 0.00411 res3b_branch2b 904367 0.00411 res3b 104886 0.00048 res4a_branch2a 485682 0.00221 res4a_branch2b 880001 0.00400 res4a_branch1 486429 0.00221 res4a 52628 0.00024 res4b_branch2a 880053 0.00400 res4b_branch2b 880035 0.00400 res4b 52478 0.00024 res5a_branch2a 1056299 0.00480 res5a_branch2b 2056857 0.00935 res5a_branch1 1056510 0.00480 res5a 26170 0.00012 res5b_branch2a 2057203 0.00935 res5b_branch2b 2057659 0.00935 res5b 26381 0.00012 pool5 71405 0.00032 fc1000 216155 0.00098

The clock frequency of the DL processor is: 220MHz

The profiler data returns these parameters and their values:

LastFrameLatency(cycles) — Total number of clock cycles for previous frame execution.
Clock frequency — Clock frequency information is retrieved from the bitstream that was used to deploy the network to the target board. For example, the profiler returns* The clock frequency of the DL processor is: 220MHz. The clock frequency of 220 MHz is retrieved from the zcu102_single bitstream.
LastFrameLatency(seconds) — Total number of seconds for previous frame execution. The total time is calculated as LastFrameLatency(cycles)/Clock Frequency. For example the conv_module LastFrameLatency(seconds) is calculated as2224115/(220*10^6).
FramesNum — Total number of input frames to the network. This value will be used in the calculation of Frames/s.
Total Latency — Total number of clock cycles to execute all the network layers and modules for FramesNum.
Frames/s — Number of frames processed in one second by the network. The total Frames/s is calculated as(FramesNum*Clock Frequency)/Total Latency. For example theFrames/s in the example is calculated as(1*220*10^6)/23659630.

The deep learning processor stores the start and end times for each layer in a memory location. Deep Learning HDL Toolbox™ extracts this information when you set the optional Profile argument for the predict method to on.

The profiler layer report does not include the layers that are fused by the deep learning compiler. For example, when your network contains custom activation layers that follow a convolution or fully connected layer, the activation layer is fused into the preceding convolution or fully connected layer to avoid extra latency. In such cases, the activation layer latency does not appear in the profiler report.

When you run predictions on multiple frames, the per layer latency in the profiler report is for the last executed frame. The total network latency and frames per second are calculated for all the frames.

Profile Inference Run - MATLAB & Simulink (original) (raw)

Finished writing input activations.

Running single input activations.

See Also

Topics