[ty] Look for site-packages directories in <sys.prefix>/lib64/ as well as <sys.prefix>/lib/ on non-Windows systems by AlexWaygood · Pull Request #19978 · astral-sh/ruff (original) (raw)
Summary
Fixes astral-sh/ty#1043; fixes astral-sh/ty#257.
According to the documentation of sys.platlibdir in the standard library, Python modules should never be installed into the lib64/pythonX.Y/site-packages directory; that directory should only be used for C extensions (which can never be submodules of a Python package, only top-level modules). Empirically, however, some installers appear to put Python files into the lib64/pythonX.Y/site-packages directory rather than the lib/pythonX.Y/site-packages directory, in some situations. I'm not sure why they're doing this -- but we'll need to start looking at the lib64 directory at some point in the future anyway to solve astral-sh/ty#487, so we may as well start looking at it now.
The practical implication of this is that a virtual environment can often have two site-packages directories on Unix, and so can a system environment. This isn't too big of a change to our overall model, since there were already situations where we accumulated site-packages directories from several different environments and added them all as search paths, but it requires a little bit of a refactor to site_packages.rs.
Test Plan
I added a CLI integration test.
I looked at whether it would be possible to add a unit test for this as an mdtest or in site_packages.rs... but concluded that both would be too much pain for too little gain here. The mdtest framework abstracts away the path to the site-packages directory by using "magic" <path-to-site-packages> path segments; this allows us to write platform-independent tests for site-packages import resolution, but means that we cannot add a unit test for something like this.
It would be possible to expand the testing framework in site_packages.rs to allow for us to add a unit test there, but it would be a big diff; I don't think it's worth it for a single test.