Error "ImportError: cannot import name 'packaging' from 'pkg_resources'" when using latest setuptools version 70 · Issue #893 · aws-neuron/aws-neuron-sdk (original) (raw)

When running examples in environment where the latest setuptools version 70 is installed, you may get the following error (example is with Nemo Megatron):

[CommandExecutionError] Command exited with status: 1. Error: Traceback (most recent call last):
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
    main()
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 134, in get_requires_for_build_wheel
    return hook(config_settings)
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
    self.run_setup()
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
    exec(code, locals())
  File "<string>", line 10, in <module>
  File "/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 25, in <module>
    from pkg_resources import packaging  # type: ignore[attr-defined] 
ImportError: cannot import name 'packaging' from 'pkg_resources' (/home/ubuntu/aws_neuron_venv_pytorch/lib/python3.8/site-packages/pkg_resources/__init__.py)

This is due to deprecation of pkg_resources API from setuptools since version 68: https://setuptools.pypa.io/en/latest/pkg_resources.html

The work-around is to pin setuptools version to before version 70:

pip install setuptools==69.5.1