Release Content — AWS Neuron Documentation (original) (raw)
Contents
This document is relevant for: Inf1, Inf2, Trn1, Trn2
Release Content#
Table of contents
Neuron 2.23.0 (05/20/2025)#
Trn1 packages#
List of packages in Neuron 2.23.0:
Component Package
Collective Communication Library aws-neuronx-collectives-2.25.65.0
Driver aws-neuronx-dkms-2.21.37.0
CustomOps aws-neuronx-gpsimd-customop-lib-0.15.12.0
CustomOps Tools aws-neuronx-gpsimd-tools-0.15.1.0
Kubernetes Plugin aws-neuronx-k8-plugin-2.25.24.0
Kubernetes Scheduler aws-neuronx-k8-scheduler-2.25.24.0
OCI aws-neuronx-oci-hook-2.9.88.0
General aws-neuronx-runtime-discovery-2.9
Runtime Library aws-neuronx-runtime-lib-2.25.57.0
System Tools aws-neuronx-tools-2.23.9.0
Jax jax_neuronx-0.5.3.1.0.719
Framework libneuronxla-2.2.3493.0
Compiler neuronx-cc-2.18.121.0
Compiler neuronx-cc-stubs-2.18.121.0
Neuron Distributed neuronx_distributed-0.12.12111
Neuron Distributed Training neuronx_distributed_training-1.3.0
Neuron Distributed Inference neuronx_distributed_inference-0.3.5591
TensorBoard tensorboard-plugin-neuronx-2.0.670.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.10.1.2.12.2.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.9.3.2.12.2.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.8.4.2.12.2.0
TensorFlow tensorflow-neuronx-2.10.1.2.1.0
PyTorch torch-neuronx-2.5.1.2.7.5413
PyTorch torch-neuronx-2.6.0.2.7.5413
PyTorch torch_xla-2.1.6
Transformers Neuron transformers-neuronx-0.13.798
Trn2 packages#
List of packages in Neuron 2.23.0:
Component Package
Collective Communication Library aws-neuronx-collectives-2.25.65.0
Driver aws-neuronx-dkms-2.21.37.0
CustomOps aws-neuronx-gpsimd-customop-lib-0.15.12.0
CustomOps Tools aws-neuronx-gpsimd-tools-0.15.1.0
Kubernetes Plugin aws-neuronx-k8-plugin-2.25.24.0
Kubernetes Scheduler aws-neuronx-k8-scheduler-2.25.24.0
OCI aws-neuronx-oci-hook-2.9.88.0
General aws-neuronx-runtime-discovery-2.9
Runtime Library aws-neuronx-runtime-lib-2.25.57.0
System Tools aws-neuronx-tools-2.23.9.0
Compiler neuronx-cc-2.18.121.0
Compiler neuronx-cc-stubs-2.18.121.0
Neuron Distributed neuronx_distributed-0.12.12111
Neuron Distributed Inference neuronx_distributed_inference-0.3.5591
PyTorch torch-neuronx-2.5.1.2.7.5413
PyTorch torch-neuronx-2.6.0.2.7.5413
PyTorch torch_xla-2.1.6
Inf2 packages#
List of packages in Neuron 2.23.0:
Component Package
Collective Communication Library aws-neuronx-collectives-2.25.65.0
Driver aws-neuronx-dkms-2.21.37.0
CustomOps aws-neuronx-gpsimd-customop-lib-0.15.12.0
CustomOps Tools aws-neuronx-gpsimd-tools-0.15.1.0
Kubernetes Plugin aws-neuronx-k8-plugin-2.25.24.0
Kubernetes Scheduler aws-neuronx-k8-scheduler-2.25.24.0
OCI aws-neuronx-oci-hook-2.9.88.0
General aws-neuronx-runtime-discovery-2.9
Runtime Library aws-neuronx-runtime-lib-2.25.57.0
System Tools aws-neuronx-tools-2.23.9.0
Jax jax_neuronx-0.5.3.1.0.719
Framework libneuronxla-2.2.3493.0
Compiler neuronx-cc-2.18.121.0
Compiler neuronx-cc-stubs-2.18.121.0
Neuron Distributed neuronx_distributed-0.12.12111
Neuron Distributed Inference neuronx_distributed_inference-0.3.5591
TensorBoard tensorboard-plugin-neuronx-2.0.670.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.10.1.2.12.2.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.9.3.2.12.2.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.8.4.2.12.2.0
TensorFlow tensorflow-neuronx-2.10.1.2.1.0
PyTorch torch-neuronx-2.5.1.2.7.5413
PyTorch torch-neuronx-2.6.0.2.7.5413
PyTorch torch_xla-2.1.6
Transformers Neuron transformers-neuronx-0.13.798
Inf1 packages#
List of packages in Neuron 2.23.0:
Component Package
Driver aws-neuronx-dkms-2.21.37.0
Kubernetes Plugin aws-neuronx-k8-plugin-2.25.24.0
Kubernetes Scheduler aws-neuronx-k8-scheduler-2.25.24.0
OCI aws-neuronx-oci-hook-2.9.88.0
System Tools aws-neuronx-tools-2.23.9.0
Compiler dmlc_nnvm-1.19.6.0
Compiler dmlc_topi-1.19.6.0
Compiler dmlc_tvm-1.19.6.0
Compiler inferentia_hwm-1.17.6.0
MXNet mx_neuron-1.8.0.2.4.147.0
MXNet mxnet_neuron-1.5.1.1.10.0.0
Compiler neuron-cc-1.24.0.0
Perf Tools neuronperf-1.8.93.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.10.1.2.12.2.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.9.3.2.12.2.0
TensorFlow Model Server tensorflow-model-server-neuronx-2.8.4.2.12.2.0
TensorFlow tensorflow-neuron-2.10.1.2.12.2.0
PyTorch torch-neuron-1.13.1.2.11.13.0
Supported Python Versions for Inf1 packages#
List of packages in Neuron 2.23.0:
Package Supported Python Versions
dmlc_nnvm-1.19.6.0 3.9, 3.10
dmlc_topi-1.19.6.0 3.9, 3.10
dmlc_tvm-1.19.6.0 3.9, 3.10
inferentia_hwm-1.17.6.0 3.9, 3.10
mx_neuron-1.8.0.2.4.147.0 3.9, 3.10
mxnet_neuron-1.5.1.1.10.0.0 3.9, 3.10
neuron-cc-1.24.0.0 3.9, 3.10
neuronperf-1.8.93.0 3.9, 3.10
tensorflow-neuron-2.10.1.2.12.2.0 3.9, 3.10
torch-neuron-1.13.1.2.11.13.0 3.9, 3.10
Supported Python Versions for Inf2/Trn1/Trn2 packages#
List of packages in Neuron 2.23.0:
Package Supported Python Versions
aws-neuronx-runtime-discovery-2.9 3.9, 3.10, 3.11
jax_neuronx-0.5.3.1.0.719 3.9
libneuronxla-2.2.3493.0 3.9, 3.10, 3.11
neuronx-cc-2.18.121.0 3.9, 3.10, 3.11
neuronx-cc-stubs-2.18.121.0 3.9, 3.10
neuronx_distributed-0.12.12111 3.9, 3.10, 3.11
neuronx_distributed_inference-0.3.5591 3.9, 3.10, 3.11
tensorflow-neuronx-2.10.1.2.1.0 3.9, 3.10
torch-neuronx-2.5.1.2.7.5413 3.9, 3.10, 3.11
torch-neuronx-2.6.0.2.7.5413 3.9, 3.10, 3.11
torch_xla-2.1.6 3.9, 3.10, 3.11
transformers-neuronx-0.13.798 3.9, 3.10, 3.11
Supported Numpy Versions#
Neuron supports versions >= 1.21.6 and <= 1.22.2
Supported HuggingFace Transformers Versions#
| Package | Supported HuggingFace Transformers Versions |
|---|---|
| torch-neuronx | < 4.35 and >=4.37.2 |
| transformers-neuronx | >= 4.36.0 |
| neuronx-distributed - Llama model class | 4.31 |
| neuronx-distributed - GPT NeoX model class | 4.26 |
| neuronx-distributed - Bert model class | 4.26 |
| nemo-megatron | 4.31.0 |
Supported Probuf Versions#
| Package | Supported Probuf versions |
|---|---|
| neuronx-cc | > 3 |
| torch-neuronx | >= 3.20 |
| torch-neuron | < 3.20 |
| transformers-neuronx | >= 3.20 |
| neuronx-distributed | >= 3.20 |
| tensorflow-neuronx | < 3.20 |
| tensorflow-neuron | < 3.20 |
Previous Neuron Releases Content#
This document is relevant for: Inf1, Inf2, Trn1, Trn2