Migrate locations.py from distutils to sysconfig · Issue #9617 · pypa/pip (original) (raw)

Hi there! If you're here from a message that's something like:

WARNING: Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/9617

Please upgrade to the newest pip release (21.1.1 at the time of writing).

- @pradyunsg (on 3-May-2021)

First off, this specific class of warnings can be safely ignored. They were added to help pip's maintainers identify incorrectly patched/configured Python distributions provided by redistributors (eg: with your OS, or as part of $corp's VM/Docker images etc). You can read more about actionable steps for end-users, in this StackOverflow answer.

Secondly, if you came here to report the message: thank you! This issue has been locked now, since this warning has been significantly more widespread than we'd expected -- in other words there are lots of Python installations that are subtly wrong in a way that hasn't mattered till today. This warning has helped surface that and enables us to plan for a smoother transition in the future.

Thirdly, we've identified certain cases where the warning is overzealous, and we'll fix that in a bugfix release soon.

Finally, I encourage you to subscribe to this issue if you want to stay up to date with this issue. We'll be discussing how we're going to handle the various edge cases reported by the users, and also announce it here once there's a release containing those fixes.

- @pradyunsg (on 26-Apr-2021)

Assuming normal CPython deprecation and release cycles, distutils will be deprecated in 3.10 and removed in 3.12. This gives us a bit more than two years to make the transition happen. Unfortunately there are a lot of wanky downstreams we need to help transition correctly in the two years.

Fortunately, logic ins location.py is relatively cheap, so we can probably try to detect issues by calculating the locations from both sources, and warn when they don’t match. This will hopefully prompt users to find issues for us that we can forward to downstreams.

So the plan is:

  1. Build a locations.py implementation that’s based entirely on sysconfig. Maintain the two implementations in parallel.
  2. When a value is requested (e.g. locations.distutils_scheme()), check whether the two implementations produce the same value. Emit a warning if they don’t (and tell the user to report this). Return the value from the old implementation.
  3. After a while, switch to return value to the new implementation. Keep the old value accessible via --use-deprecated. Keep warning when the two values don’t match.
  4. Drop the old implementation (and the warning) when distutils is formally dropped in CPython master.