Neuron Check Model — AWS Neuron Documentation (original) (raw)

This document is relevant for: Inf1

Neuron Check Model#

Overview#

Neuron Check Model tool provides user with basic information about the compiled and uncompiled model’s operations without the use of TensorBoard-Neuron. For additional visibility into the models, please see Neuron Plugin for TensorBoard (Inf1).

Neuron Check Model tool scans the user’s uncompiled model and provides a table of the operations within the uncompiled model. By default, the table shows each operation type and number of instances of that type within model, and whether the type is supported in Neuron. If –show_names option is specified, the table shows each operation by name and whether the type of that operation is supported in Neuron.

If the model is already compiled, the tool also provides the table of operations as for uncompiled model. The table include the Neuron subgraph type and number of instances of that type, along with operations that have not been compiled to Neuron. Additionally, the tool displays a message showing the minimum number of NeuronCores required to run the model, followed by another table which shows the list of Neuron subgraphs by name and the number of pipelined NeuronCores used by each subgraph. More information about NeuronCore pipeline can be found inNeuronCore Pipeline. If –expand_subgraph option is specified, the operations within each subgraph are printed below the subgraph information.

Neuron Check Model tool is currently available for TensorFlow and MXNet. To check PT model, please use torch.neuron.analyze_model function as shown in PyTorch-Neuron Getting Started tutorial ResNet50 model for Inferentia

TensorFlow-Neuron Check Model#

The following example shows how to run TensorFlow-Neuron Check Model tool with TensorFlow ResNet50 tutorial.

1. Start with the TensorFlow ResNet50 tutorial at Running ResNet50 on Inferentia and do the first three steps of the tutorial. Please stay in the Python environment that you setup during the tutorial.

  1. Install needed tensorflow_hub package and download the tool:

pip install tensorflow_hub wget https://raw.githubusercontent.com/aws/aws-neuron-sdk/master/src/neuron-gatherinfo/tf_neuron_check_model.py python tf_neuron_check_model.py -h

usage: tf_neuron_check_model.py [-h] [--show_names] [--expand_subgraph] model_path

positional arguments: model_path a TensorFlow SavedModel directory (currently supporting TensorFlow v1 SaveModel only).

optional arguments: -h, --help show this help message and exit --show_names list operation by name instead of summarizing by type (caution: this option will generate many lines of output for a large model). --expand_subgraph show subgraph operations.

  1. After step 3 of the TensorFlow ResNet50 tutorial, you can check the uncompiled model to see Neuron supported operations (currently supporting TensorFlow v1 SaveModel only):

$ python tf_neuron_check_model.py ws_resnet50/resnet50/

Op Type Num Instances Neuron Supported ?

Pad 2 Yes RandomUniform 54 Yes Sub 54 Yes Mul 54 Yes Add 54 Yes Conv2D 53 Yes BiasAdd 54 Yes FusedBatchNormV3 53 Yes Relu 49 Yes MaxPool 1 Yes AddV2 16 Yes Fill 56 Yes Mean 1 Yes MatMul 1 Yes Softmax 1 Yes Pack 1 Yes

  1. You can also check the compiled model to see the number of pipeline NeuronCores for each subgraph:

$ python tf_neuron_check_model.py ws_resnet50/resnet50_neuron/

Op Type Num Instances Neuron Supported ?

NeuronOp 1 Yes

Subgraph Name Num Pipelined NeuronCores

conv5_block3_3_bn/FusedBatchNormV3/ReadVariableOp/neuron_op_d6f098c01c780733 1

  1. When showing subgraph information, you can use –expand_subgraph to show operation types in each subgraph:

$ python tf_neuron_check_model.py ws_resnet50/resnet50_neuron/ --expand_subgraph

(output truncated to show subgraph information only)

Subgraph Name Num Pipelined NeuronCores

conv5_block3_3_bn/FusedBatchNormV3/ReadVariableOp/neuron_op_d6f098c01c780733 1 Op Type Num Instances ------- ------------- MatMul 1 Relu 49 Add 16 FusedBatchNorm 53 BiasAdd 54 Conv2D 53 Pad 2 Mean 1 MaxPool 1 Softmax 1

  1. Use –show_names to see full operation names (caution: this option will generate many lines of output for a large model):

$ python tf_neuron_check_model.py ws_resnet50/resnet50_neuron/ --show_names

Op Name Op Type Neuron Supported ?

conv5_block3_3_bn/FusedBatchNormV3/ReadVariableOp/neuron_op_d6f098c01c780733 NeuronOp Yes

Subgraph Name Num Pipelined NeuronCores

conv5_block3_3_bn/FusedBatchNormV3/ReadVariableOp/neuron_op_d6f098c01c780733 1

MXNet-Neuron Check Model#

The following example shows how to run MXNet-Neuron Check Model tool with MXNet ResNet50 tutorial.

1. Start with the MXNet ResNet50 tutorial at Running Neuron Apache MXNet ResNet50 on Inferentia and do the first three steps of the tutorial. Please stay in the Python environment that you setup during the tutorial.

  1. Download the tool:

wget https://raw.githubusercontent.com/aws/aws-neuron-sdk/master/src/neuron-gatherinfo/mx_neuron_check_model.py python mx_neuron_check_model.py -h

usage: mx_neuron_check_model.py [-h] [--show_names] [--expand_subgraph] model_path

positional arguments: model_path path prefix to MXNet model (the part before -symbol.json)

optional arguments: -h, --help show this help message and exit --show_names list operation by name instead of summarizing by type (caution: this option will generate many lines of output for a large model). --expand_subgraph show subgraph operations.

  1. After step 3 of MXNet ResNet50 tutorial, you can check the uncompiled model to see Neuron supported operations:

$ python mx_neuron_check_model.py resnet-50

Op Type Num Instances Neuron Supported ?

BatchNorm 51 Yes Convolution 53 Yes Activation 50 Yes Pooling 2 Yes elemwise_add 16 Yes Flatten 1 Yes FullyConnected 1 Yes SoftmaxOutput 1 No

  1. You can also check the compiled model to see the number of pipeline NeuronCores for each subgraph:

$ python mx_neuron_check_model.py resnet-50_compiled

Op Type Num Instances Neuron Supported ?

_neuron_subgraph_op 1 Yes SoftmaxOutput 1 No

Subgraph Name Num Pipelined NeuronCores

_neuron_subgraph_op0 1

  1. When showing subgraph information, you can use –expand_subgraph to show operation types in each subgraph:

$ python mx_neuron_check_model.py resnet-50_compiled --expand_subgraph

(output truncated to show subgraph information only)

Subgraph Name Num Pipelined NeuronCores

_neuron_subgraph_op0 1 Op Type Num Instances ------- ------------- BatchNorm 51 Convolution 53 Activation 50 Pooling 2 elemwise_add 16 Flatten 1 FullyConnected 1

  1. Use –show_names to see full operation names (caution: this option will generate many lines of output for a large model):

$ python mx_neuron_check_model.py resnet-50_compiled --show_names

Op Name Op Type Neuron Supported ?

_neuron_subgraph_op0 _neuron_subgraph_op Yes softmax SoftmaxOutput No

Subgraph Name Num Pipelined NeuronCores

_neuron_subgraph_op0 1

This document is relevant for: Inf1