Estimate Performance of Deep Learning Network - MATLAB & Simulink (original) (raw)

To reduce the time required to design a custom deep learning network that meets performance requirements, before deploying the network, analyze layer level latencies. Compare deep learning network performances on custom bitstream processor configurations to performances on reference (shipping) bitstream processor configurations.

To learn how to use the information in the table data from theestimatePerformance function to calculate your network performance, seeProfile Inference Run.

Estimate Performance of Custom Deep Learning Network for Custom Processor Configuration

This example shows how to calculate the performance of a deep learning network for a custom processor configuration.

1. Create a file in your current working folder calledgetLogoNetwork.m. In the file, enter:
   function net = getLogoNetwork()
   if ~isfile('LogoNet.mat')
   url = 'https://www.mathworks.com/supportfiles/gpucoder/cnn_models/logo_detection/LogoNet.mat';
   websave('LogoNet.mat',url);
   end
   data = load('LogoNet.mat');
   net = data.convnet;
   end
   Call the function and save the result in snet.
2. Create a dlhdl.ProcessorConfig object.
   hPC = dlhdl.ProcessorConfig;
3. Call estimatePerformance with snet to retrieve the layer level latencies and performance for the LogoNet network.
   hPC.estimatePerformance(snet)
   3 Memory Regions created.
   Deep Learning Processor Estimator Performance Results
   LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s
   ------------- ------------- --------- --------- ---------

Network 39853460 0.19927 1 39853460 5.0
conv_1 6825287 0.03413
maxpool_1 3755088 0.01878
conv_2 10440701 0.05220
maxpool_2 1447840 0.00724
conv_3 9393397 0.04697
maxpool_3 1765856 0.00883
conv_4 1770484 0.00885
maxpool_4 28098 0.00014
fc_1 2644884 0.01322
fc_2 1692532 0.00846
fc_3 89293 0.00045

• The clock frequency of the DL processor is: 200MHz

Evaluate Performance of Deep Learning Network on Custom Processor Configuration

Benchmark the performance of a deep learning network on a custom bitstream configuration by comparing it to the performance on a reference (shipping) bitstream configuration. Use the comparison results to adjust your custom deep learning processor parameters to achieve optimum performance.

In this example compare the performance of the ResNet-18 network on the zcu102_single bitstream configuration to the performance on the default custom bitstream configuration.

Load Pretrained Network

Load the pretrained network.

Retrieve zcu102_single Bitstream Configuration

To retrieve the zcu102_single bitstream configuration, use the dlhdl.ProcessorConfig object. For more information, see dlhdl.ProcessorConfig. To learn about modifiable parameters of the processor configuration, see getModuleProperty and setModuleProperty.

hPC_shipping = dlhdl.ProcessorConfig('Bitstream',"zcu102_single")

hPC_shipping = Processing Module "conv" ModuleGeneration: 'on' LRNBlockGeneration: 'on' ConvThreadNumber: 16 InputMemorySize: [227 227 3] OutputMemorySize: [227 227 3] FeatureSizeLimit: 2048

                  Processing Module "fc"
                        ModuleGeneration: 'on'
                  SoftmaxBlockGeneration: 'off'
                          FCThreadNumber: 4
                         InputMemorySize: 25088
                        OutputMemorySize: 4096

               Processing Module "adder"
                        ModuleGeneration: 'on'
                         InputMemorySize: 40
                        OutputMemorySize: 40

          Processor Top Level Properties
                          RunTimeControl: 'register'
                      InputDataInterface: 'External Memory'
                     OutputDataInterface: 'External Memory'
                       ProcessorDataType: 'single'

                 System Level Properties
                          TargetPlatform: 'Xilinx Zynq UltraScale+ MPSoC ZCU102 Evaluation Kit'
                         TargetFrequency: 220
                           SynthesisTool: 'Xilinx Vivado'
                         ReferenceDesign: 'AXI-Stream DDR Memory Access : 3-AXIM'
                 SynthesisToolChipFamily: 'Zynq UltraScale+'
                 SynthesisToolDeviceName: 'xczu9eg-ffvb1156-2-e'
                SynthesisToolPackageName: ''
                 SynthesisToolSpeedValue: ''

Estimate ResNet-18 Performance for zcu102_single Bitstream Configuration

To estimate the performance of the ResNet-18 DAG network, use the estimatePerformance function of the dlhdl.ProcessorConfig object. The function returns the estimated layer latency, network latency, and network performance in frames per second (Frames/s).

hPC_shipping.estimatePerformance(snet)

Optimizing series network: Fused 'nnet.cnn.layer.BatchNormalizationLayer' into 'nnet.cnn.layer.Convolution2DLayer'

Notice: The layer 'data' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software.

Notice: The layer 'ClassificationLayer_predictions' with type 'nnet.cnn.layer.ClassificationOutputLayer' is implemented in software.

          Deep Learning Processor Estimator Performance Results

               LastFrameLatency(cycles)   LastFrameLatency(seconds)       FramesNum      Total Latency     Frames/s
                     -------------             -------------              ---------        ---------       ---------

Network 23634966 0.10743 1 23634966 9.3 ____conv1 2165372 0.00984 ____pool1 646226 0.00294 ____res2a_branch2a 966221 0.00439 ____res2a_branch2b 966221 0.00439 ____res2a 210750 0.00096 ____res2b_branch2a 966221 0.00439 ____res2b_branch2b 966221 0.00439 ____res2b 210750 0.00096 ____res3a_branch1 540749 0.00246 ____res3a_branch2a 763860 0.00347 ____res3a_branch2b 919117 0.00418 ____res3a 105404 0.00048 ____res3b_branch2a 919117 0.00418 ____res3b_branch2b 919117 0.00418 ____res3b 105404 0.00048 ____res4a_branch1 509261 0.00231 ____res4a_branch2a 509261 0.00231 ____res4a_branch2b 905421 0.00412 ____res4a 52724 0.00024 ____res4b_branch2a 905421 0.00412 ____res4b_branch2b 905421 0.00412 ____res4b 52724 0.00024 ____res5a_branch1 1046605 0.00476 ____res5a_branch2a 1046605 0.00476 ____res5a_branch2b 2005197 0.00911 ____res5a 26368 0.00012 ____res5b_branch2a 2005197 0.00911 ____res5b_branch2b 2005197 0.00911 ____res5b 26368 0.00012 ____pool5 54594 0.00025 ____fc1000 207852 0.00094

• The clock frequency of the DL processor is: 220MHz

Create Custom Processor Configuration

To create a custom processor configuration, use the dlhdl.ProcessorConfig object. For more information, see dlhdl.ProcessorConfig. To learn about modifiable parameters of the processor configuration, see getModuleProperty and setModuleProperty.

hPC_custom = dlhdl.ProcessorConfig

hPC_custom = Processing Module "conv" ModuleGeneration: 'on' LRNBlockGeneration: 'on' ConvThreadNumber: 16 InputMemorySize: [227 227 3] OutputMemorySize: [227 227 3] FeatureSizeLimit: 2048

                  Processing Module "fc"
                        ModuleGeneration: 'on'
                  SoftmaxBlockGeneration: 'off'
                          FCThreadNumber: 4
                         InputMemorySize: 25088
                        OutputMemorySize: 4096

               Processing Module "adder"
                        ModuleGeneration: 'on'
                         InputMemorySize: 40
                        OutputMemorySize: 40

          Processor Top Level Properties
                          RunTimeControl: 'register'
                      InputDataInterface: 'External Memory'
                     OutputDataInterface: 'External Memory'
                       ProcessorDataType: 'single'

                 System Level Properties
                          TargetPlatform: 'Xilinx Zynq UltraScale+ MPSoC ZCU102 Evaluation Kit'
                         TargetFrequency: 200
                           SynthesisTool: 'Xilinx Vivado'
                         ReferenceDesign: 'AXI-Stream DDR Memory Access : 3-AXIM'
                 SynthesisToolChipFamily: 'Zynq UltraScale+'
                 SynthesisToolDeviceName: 'xczu9eg-ffvb1156-2-e'
                SynthesisToolPackageName: ''
                 SynthesisToolSpeedValue: ''

Estimate ResNet-18 Performance for Custom Bitstream Configuration

To estimate the performance of the ResNet-18 DAG network, use the estimatePerformance function of the dlhdl.ProcessorConfig object. The function returns the estimated layer latency, network latency, and network performance in frames per second (Frames/s).

hPC_custom.estimatePerformance(snet)

Optimizing series network: Fused 'nnet.cnn.layer.BatchNormalizationLayer' into 'nnet.cnn.layer.Convolution2DLayer'

Notice: The layer 'data' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software.

Notice: The layer 'ClassificationLayer_predictions' with type 'nnet.cnn.layer.ClassificationOutputLayer' is implemented in software.

          Deep Learning Processor Estimator Performance Results

               LastFrameLatency(cycles)   LastFrameLatency(seconds)       FramesNum      Total Latency     Frames/s
                     -------------             -------------              ---------        ---------       ---------

Network 21219873 0.10610 1 21219873 9.4 ____conv1 2165372 0.01083 ____pool1 646226 0.00323 ____res2a_branch2a 966221 0.00483 ____res2a_branch2b 966221 0.00483 ____res2a 210750 0.00105 ____res2b_branch2a 966221 0.00483 ____res2b_branch2b 966221 0.00483 ____res2b 210750 0.00105 ____res3a_branch1 540749 0.00270 ____res3a_branch2a 708564 0.00354 ____res3a_branch2b 919117 0.00460 ____res3a 105404 0.00053 ____res3b_branch2a 919117 0.00460 ____res3b_branch2b 919117 0.00460 ____res3b 105404 0.00053 ____res4a_branch1 509261 0.00255 ____res4a_branch2a 509261 0.00255 ____res4a_branch2b 905421 0.00453 ____res4a 52724 0.00026 ____res4b_branch2a 905421 0.00453 ____res4b_branch2b 905421 0.00453 ____res4b 52724 0.00026 ____res5a_branch1 751693 0.00376 ____res5a_branch2a 751693 0.00376 ____res5a_branch2b 1415373 0.00708 ____res5a 26368 0.00013 ____res5b_branch2a 1415373 0.00708 ____res5b_branch2b 1415373 0.00708 ____res5b 26368 0.00013 ____pool5 54594 0.00027 ____fc1000 207351 0.00104

• The clock frequency of the DL processor is: 200MHz

The performance of the ResNet-18 network on the custom bitstream configuration is lower than the performance on the zcu102_single bitstream configuration. The difference between the custom bitstream configuration and the zcu102_single bitstream configuration is the target frequency.

Modify Custom Processor Configuration

Modify the custom processor configuration to increase the target frequency. To learn about modifiable parameters of the processor configuration, see dlhdl.ProcessorConfig.

hPC_custom.TargetFrequency = 220; hPC_custom

hPC_custom = Processing Module "conv" ModuleGeneration: 'on' LRNBlockGeneration: 'on' ConvThreadNumber: 16 InputMemorySize: [227 227 3] OutputMemorySize: [227 227 3] FeatureSizeLimit: 2048

                  Processing Module "fc"
                        ModuleGeneration: 'on'
                  SoftmaxBlockGeneration: 'off'
                          FCThreadNumber: 4
                         InputMemorySize: 25088
                        OutputMemorySize: 4096

               Processing Module "adder"
                        ModuleGeneration: 'on'
                         InputMemorySize: 40
                        OutputMemorySize: 40

          Processor Top Level Properties
                          RunTimeControl: 'register'
                      InputDataInterface: 'External Memory'
                     OutputDataInterface: 'External Memory'
                       ProcessorDataType: 'single'

                 System Level Properties
                          TargetPlatform: 'Xilinx Zynq UltraScale+ MPSoC ZCU102 Evaluation Kit'
                         TargetFrequency: 220
                           SynthesisTool: 'Xilinx Vivado'
                         ReferenceDesign: 'AXI-Stream DDR Memory Access : 3-AXIM'
                 SynthesisToolChipFamily: 'Zynq UltraScale+'
                 SynthesisToolDeviceName: 'xczu9eg-ffvb1156-2-e'
                SynthesisToolPackageName: ''
                 SynthesisToolSpeedValue: ''

Re-estimate ResNet-18 Performance for Modified Custom Bitstream Configuration

Estimate the performance of the ResNet-18 DAG network on the modified custom bitstream configuration.

hPC_custom.estimatePerformance(snet)

Optimizing series network: Fused 'nnet.cnn.layer.BatchNormalizationLayer' into 'nnet.cnn.layer.Convolution2DLayer'

Notice: The layer 'data' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software.

Notice: The layer 'ClassificationLayer_predictions' with type 'nnet.cnn.layer.ClassificationOutputLayer' is implemented in software.

          Deep Learning Processor Estimator Performance Results

               LastFrameLatency(cycles)   LastFrameLatency(seconds)       FramesNum      Total Latency     Frames/s
                     -------------             -------------              ---------        ---------       ---------

Network 23634966 0.10743 1 23634966 9.3 ____conv1 2165372 0.00984 ____pool1 646226 0.00294 ____res2a_branch2a 966221 0.00439 ____res2a_branch2b 966221 0.00439 ____res2a 210750 0.00096 ____res2b_branch2a 966221 0.00439 ____res2b_branch2b 966221 0.00439 ____res2b 210750 0.00096 ____res3a_branch1 540749 0.00246 ____res3a_branch2a 763860 0.00347 ____res3a_branch2b 919117 0.00418 ____res3a 105404 0.00048 ____res3b_branch2a 919117 0.00418 ____res3b_branch2b 919117 0.00418 ____res3b 105404 0.00048 ____res4a_branch1 509261 0.00231 ____res4a_branch2a 509261 0.00231 ____res4a_branch2b 905421 0.00412 ____res4a 52724 0.00024 ____res4b_branch2a 905421 0.00412 ____res4b_branch2b 905421 0.00412 ____res4b 52724 0.00024 ____res5a_branch1 1046605 0.00476 ____res5a_branch2a 1046605 0.00476 ____res5a_branch2b 2005197 0.00911 ____res5a 26368 0.00012 ____res5b_branch2a 2005197 0.00911 ____res5b_branch2b 2005197 0.00911 ____res5b 26368 0.00012 ____pool5 54594 0.00025 ____fc1000 207852 0.00094

• The clock frequency of the DL processor is: 220MHz

See Also

dlhdl.ProcessorConfig | getModuleProperty | setModuleProperty | estimatePerformance | estimateResources

Topics

• Estimate Resource Utilization for Custom Processor Configuration
• Effects of Custom Deep Learning Processor Parameters on Performance and Resource Utilization