Running Neuron Apache MXNet ResNet50 on Inferentia — AWS Neuron Documentation (original) (raw)

Contents

• Introduction:
  • Warning
• Compile model on Neuron
• Deploy on Inferentia

Running Neuron Apache MXNet ResNet50 on Inferentia#

Introduction:#

In this tutorial we will compile and deploy ResNet50 model for Inferentia. In this tutorial we provide two main sections:

1.Compile the ResNet50 model.

2.Infer the compiled model.

Before running the following verify this Jupyter notebook is running “conda_aws_neuron_mxnet_p36” kernel. You can select the Kernel from the “Kernel -> Change Kernel” option on the top of this Jupyter notebook page. Neuron supports Python module, Symbol APIs and the C predict API. The following quick start example uses the Symbol API.

Warning#

This tutorial was tested on MXNet-1.5

MXNet-1.5 entered maintenance mode and require Neuron runtime 1.0, please see : MXNet-1.5 enters maintainence mode

To setup development environment for MXNet-1.5 see installation instructions for Neuron 1.15.1 : Neuron-1.15.1 MXNet install

Compile model on Neuron#

The following step will compile the resnet50 model. Compilation will take a few minutes on inf1.6xlarge. At the end of compilation, the files resnet-50_compiled-0000.params and resnet-50_compiled-symbol.json will be created in local directory.

import mxnet as mx import numpy as np

path='http://data.mxnet.io/models/imagenet/' mx.test_utils.download(path+'resnet/50-layers/resnet-50-0000.params') mx.test_utils.download(path+'resnet/50-layers/resnet-50-symbol.json') sym, args, aux = mx.model.load_checkpoint('resnet-50', 0)

Compile for Inferentia using Neuron

inputs = { "data" : mx.nd.ones([1,3,224,224], name='data', dtype='float32') } sym, args, aux = mx.contrib.neuron.compile(sym, args, aux, inputs)

#save compiled model mx.model.save_checkpoint("resnet-50_compiled", 0, sym, args, aux)

Deploy on Inferentia#

Using same instance to deploy the model.

import mxnet as mx import numpy as np

path='http://data.mxnet.io/models/imagenet/' mx.test_utils.download(path+'synset.txt')

fname = mx.test_utils.download('https://raw.githubusercontent.com/awslabs/mxnet-model-server/master/docs/images/kitten_small.jpg?raw=true') img = mx.image.imread(fname)# convert into format (batch, RGB, width, height) img = mx.image.imresize(img, 224, 224) # resize img = img.transpose((2, 0, 1)) # Channel first img = img.expand_dims(axis=0) # batchify img = img.astype(dtype='float32')

sym, args, aux = mx.model.load_checkpoint('resnet-50_compiled', 0) softmax = mx.nd.random_normal(shape=(1,)) args['softmax_label'] = softmax args['data'] = img

Inferentia context

ctx = mx.neuron()

exe = sym.bind(ctx=ctx, args=args, aux_states=aux, grad_req='null')

with open('synset.txt', 'r') as f: labels = [l.rstrip() for l in f]

exe.forward(data=img) prob = exe.outputs[0].asnumpy()# print the top-5 prob = np.squeeze(prob) a = np.argsort(prob)[::-1] for i in a[0:5]: print('probability=%f, class=%s' %(prob[i], labels[i]))

Sample output will look like below:

#probability=0.634792, class=n02123045 tabby, tabby cat #probability=0.193601, class=n02123159 tiger cat #probability=0.103627, class=n02124075 Egyptian cat #probability=0.031604, class=n02127052 lynx, catamount #probability=0.015892, class=n02129604 tiger, Panthera tigris