TensorFlow 2.x (tensorflow-neuron) Release Notes — AWS Neuron Documentation (original) (raw)

TensorFlow 2.x (tensorflow-neuron) Release Notes#

Table of contents

• Known Issues and Limitations - updated 08/12/2021
• tensorflow-neuron 2.x release [2.12.2.0]
• tensorflow-neuron 2.x release [2.11.4.0]
• tensorflow-neuron 2.x release [2.10.19.0]
• tensorflow-neuron 2.x release [2.10.8.0]
• tensorflow-neuron 2.x release [2.10.2.0]
• tensorflow-neuron 2.x release [2.10.1.0]
• tensorflow-neuron 2.x release [2.9.3.0]
• tensorflow-neuron 2.x release [2.8.9.0]
• tensorflow-neuron 2.x release [2.8.1.0]
• tensorflow-neuron 2.x release [2.7.4.0]
• tensorflow-neuron 2.x release [2.7.3.0]
• tensorflow-neuron 2.x release [2.6.5.0]
• tensorflow-neuron 2.x release [2.6.0.0]
• tensorflow-neuron 2.x release [2.4.0.0]
• tensorflow-neuron 2.x release [2.3.0.0]
• tensorflow-neuron 2.x release [2.2.0.0]
• tensorflow-neuron 2.x release [2.1.14.0]
• tensorflow-neuron 2.x release [2.1.13.0]
• tensorflow-neuron 2.x release [2.1.6.0]
• tensorflow-neuron 2.x release [2.0.4.0]
• tensorflow-neuron 2.x release [2.0.3.0]
• tensorflow-neuron 2.x release [1.6.8.0]

This document lists the release notes for the tensorflow-neuron 2.x packages.

Known Issues and Limitations - updated 08/12/2021#

• Support on serialized TensorFlow 2.x custom operators is currently limited. Serializing some operators registered from tensorflow-text through TensorFlow Hub is going to cause failure in tensorflow.neuron.trace.
• Memory leak exists on latest releases of TensorFlow Neuron for versions 2.1, 2.2, 2.3, and 2.4.
• Issue: When compiling large models, user might run out of memory and encounter this fatal error.

terminate called after throwing an instance of 'std::bad_alloc'

Solution: run compilation on a c5.4xlarge instance type or larger.

• Issue: When upgrading tensorflow-neuron withpip install tensorflow-neuron --upgrade, the following error message may appear, which is caused by pip version being too low.

Could not find a version that satisfies the requirement tensorflow<1.16.0,>=1.15.0 (from tensorflow-neuron)

Solution: run a pip install pip --upgrade before upgradingtensorflow-neuron.

• Issue: Some Keras routines throws the following error:

AttributeError: 'str' object has no attribute 'decode'.

Solution: Please downgrade h5py by pip install ‘h5py<3’. This is caused by TensorFlow/TensorFlow#44467.

tensorflow-neuron 2.x release [2.12.2.0]#

Date: 09/16/2024

• Minor updates.

tensorflow-neuron 2.x release [2.11.4.0]#

Date: 07/03/2024

• Minor updates.

tensorflow-neuron 2.x release [2.10.19.0]#

Date: 04/01/2024

• Minor updates.

tensorflow-neuron 2.x release [2.10.8.0]#

Date: 12/21/2023

• Minor updates.

tensorflow-neuron 2.x release [2.10.2.0]#

Date: 10/15/2023

• Minor updates.

tensorflow-neuron 2.x release [2.10.1.0]#

Date: 09/15/2023

• Minor updates.

tensorflow-neuron 2.x release [2.9.3.0]#

Date: 7/19/2023

• Minor updates.

tensorflow-neuron 2.x release [2.8.9.0]#

Date: 06/14/2023

• Added Python 3.10 support.

tensorflow-neuron 2.x release [2.8.1.0]#

Date: 05/01/2023

• Added support for tracing models larger than 2 GB through the environment variable NEURON_CC_FLAGS='--extract-weights INSTANCE_TYPE' for all inf1 instance types.
• Neuron release 2.10 release will be the last release that will include support for tensorflow-neuron version 2.7. Future Neuron releases will not include tensorflow-neuron version 2.7.

tensorflow-neuron 2.x release [2.7.4.0]#

Date: 04/19/2023

• Minor updates.

tensorflow-neuron 2.x release [2.7.3.0]#

Date: 03/28/2023

• Introduce the tfn.analyze_model function that displays information about the supported and unsupported operators of a traceable model.
• Introduce the on_neuron_ratio attribute of AWS Optimized Neuron Models returned by tfn.trace, which is the percentage of ops on neuron after compilation.

tensorflow-neuron 2.x release [2.6.5.0]#

Date: 02/24/2023

• Minor updates.

tensorflow-neuron 2.x release [2.6.0.0]#

Date: 2/24/2023

• Minor bug fixes.

tensorflow-neuron 2.x release [2.4.0.0]#

Date: 11/22/2022

• Beta support for tracing models larger than 2 GB through environment variable NEURON_CC_FLAGS='--extract-weights'.
• Introduce tfn.auto_multicore Python API to enable automatic data parallel on multiple NeuronCores.
• Introduce tf-neuron-auto-multicore tool to enable automatic data parallel on multiple NeuronCores.
• Deprecated the NEURONCORE_GROUP_SIZES environment variable.
• Minor bug fixes.

tensorflow-neuron 2.x release [2.3.0.0]#

Date: 04/29/2022

• Added support for Tensorflow 2.8.0.
• Added support for Slice operator
• The graph partitioner now prefers to place less compute intensive operators on CPU if the model already contains a large amount of compute intensive operators.
• Fixed Github issue #408, the fix solves data type handling bug in tfn.trace when the model contains Conv2D operators.

tensorflow-neuron 2.x release [2.2.0.0]#

Date: 03/25/2022

• Updated TensorFlow 2.5 to version 2.5.3.
• Added support for TensorFlow 2.6 and 2.7.
• Added a warning message when calling tfn.saved_model.compile API. In tensorflow-neuron 2.x you should call tensorflow.neuron.trace. tfn.saved_model.compile API supports only partial functionality of tensorflow.neuron.trace and will be deprecated in the future.

tensorflow-neuron 2.x release [2.1.14.0]#

Date: 02/17/2022

• Fixed a bug in TensorFlow Neuron versions 2.1, 2.2. 2.3 and 2.4. The fixed bug was causing a memory leak of 128 bytes for each inference.
• Improved warning message when calling deprecated compilation API under tensorflow-neuron 2.x.

tensorflow-neuron 2.x release [2.1.13.0]#

Date: 02/16/2022

• Fixed a bug that caused a memory leak. The memory leak was approximately 128b for each inference and exists in all versions of TensorFlow Neuron versions part of Neuron 1.16.0 to Neuron 1.17.0 releases. see Previous Releases Artifacts (Neuron 2.x)for exact versions included in each release. This release only addresses the leak in TensorFlow Neuron 2.5. Future release of TensorFlow Neuron will fix the leak in other versions as well (2.1, 2.2, 2.3, 2.4).

tensorflow-neuron 2.x release [2.1.6.0]#

Date: 01/20/2022

• Updated TensorFlow 2.5 to version 2.5.2.
• Enhanced auto data parallel (e.g. when using NEURONCORE_GROUP_SIZES=X,Y,Z,W) to support edge cases.
• Fixed a bug that may cause tensorflow-neuron to generate in some cases scalar gather instruction with incorrect arguments.

tensorflow-neuron 2.x release [2.0.4.0]#

Date: 11/05/2021

• Updated Neuron Runtime (which is integrated within this package) to libnrt 2.2.18.0 to fix a container issue that was preventing the use of containers when /dev/neuron0 was not present. See details here neuron-runtime-release-notes.

tensorflow-neuron 2.x release [2.0.3.0]#

Date: 10/27/2021

New in this release#

• TensorFlow 2.x (tensorflow-neuron) now support Neuron Runtime 2.x (libnrt.so shared library) only.

  Important

  • You must update to the latest Neuron Driver (aws-neuron-dkms version 2.1 or newer) for proper functionality of the new runtime library.
  • Read Introducing Neuron Runtime 2.x (libnrt.so)application note that describes why are we making this change and how this change will affect the Neuron SDK in detail.
  • Read Migrate your application to Neuron Runtime 2.x (libnrt.so) for detailed information of how to migrate your application.

• Updated TensorFlow 2.3.x from TensorFlow 2.3.3 to TensorFlow 2.3.4.
• Updated TensorFlow 2.4.x from TensorFlow 2.4.2 to TensorFlow 2.4.3.
• Updated TensorFlow 2.5.x from TensorFlow 2.5.0 to TensorFlow 2.5.1.

Resolved Issues#

• Fix bug that can cause illegal compiler optimizations
• Fix bug that can cause dynamic-shape operators be placed on Neuron

tensorflow-neuron 2.x release [1.6.8.0]#

Date: 08/12/2021

New in this release#

• First release of TensorFlow 2.x integration, Neuron support now TensorFlow versions 2.1.4, 2.2.3, 2.3.3, 2.4.2, and 2.5.0.
• New public API tensorflow.neuron.trace: trace a TensorFlow 2.x keras.Model or a Python callable that can be decorated by tf.function, and return an AWS-Neuron-optimized keras.Model that can execute on AWS Machine Learning Accelerators.

Please note that TensorFlow 1.x SavedModel compilation API tensorflow.neuron.saved_model.compile is not supported in tensorflow-neuron 2.x . It continues to function in tensorflow-neuron 1.15.x .

• Included versions:

  • tensorflow-neuron-2.5.0.1.6.8.0
  • tensorflow-neuron-2.4.2.1.6.8.0
  • tensorflow-neuron-2.3.3.1.6.8.0
  • tensorflow-neuron-2.2.3.1.6.8.0
  • tensorflow-neuron-2.1.4.1.6.8.0

This document is relevant for: Inf1