Use the "venv" scheme if available to obtain prefixed lib paths by dnicolodi · Pull Request #11598 · pypa/pip (original) (raw)
get_prefixed_libs() computes the Python path for libraries in a pip isolation environment. Python 3.11 introduced the "venv" path scheme to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the default paths scheme when Python is invoked outside a virtual environment is "osx_framework_library" and does not relative to the "{base}" or "{platbase}" variables.
Fixes #11539.
dnicolodi added a commit to dnicolodi/meson-python that referenced this pull request
dnicolodi added a commit to dnicolodi/meson-python that referenced this pull request
Comment on lines 216 to 222
| def get_prefixed_libs(prefix: str) -> typing.Tuple[str, str]: |
|---|
| paths = sysconfig.get_paths(vars={"base": prefix, "platbase": prefix}) |
| vars = {"base": prefix, "platbase": prefix} |
| if "venv" in sysconfig.get_scheme_names(): |
| paths = sysconfig.get_paths(vars=vars, scheme="venv") |
| else: |
| paths = sysconfig.get_paths(vars=vars) |
| return (paths["purelib"], paths["platlib"]) |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this function is only used for creating isolated environments, let’s rename it to better describe what it does. This avoids needing to think about why this always sets venv even when pip is not running in a virtual environment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought prefixed in the name carried this message. Do you have a better name in mind?
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe just get_isolated_environment_lib_paths?
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds descriptive enough. Should I change the argument name to environment then?
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
prefix is probably fine (it is the environment’s path prefix after all)
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just noticed that the function name is meant to reflect the name of a similar function in distutils. It is also called from a function with the same name in another module, see
| def get_prefixed_libs(prefix: str) -> List[str]: |
|---|
| """Return the lib locations under ``prefix``.""" |
| new_pure, new_plat = _sysconfig.get_prefixed_libs(prefix) |
| if _USE_SYSCONFIG: |
| return _deduplicated(new_pure, new_plat) |
Should we still change just this one?
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, change them all.
FFY00 pushed a commit to mesonbuild/meson-python that referenced this pull request
get_prefixed_libs() computes the Python path for libraries in a pip isolation environment. Python 3.11 introduced the "venv" path scheme to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the default paths scheme when Python is invoked outside a virtual environment is "osx_framework_library" and does not relative to the "{base}" or "{platbase}" variables.
Fixes pypa#11539.
Since this function is only used for creating isolated environments, rename it to better describe what it does. This avoids needing to think about why the implementation uses the "venv" paths scheme even when pip is not running in a virtual environment.
I've triggered a CI re-run for some flakiness.
All green now. Can this be merged or there is anything left to do?
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi added a commit to dnicolodi/pip that referenced this pull request
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes.
However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293.
The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
