[ty] Support import <namespace> and from <namespace> import module by MichaReiser · Pull Request #18137 · astral-sh/ruff (original) (raw)

Summary

The module resolver incorrectly returned None if a module name resolves to a namespace package (it did correctly handle the case where a module name resolves to a file inside a namespace package).

Supporting this requires a smaller refactor because Module assumed that file and search_path are always known. However, neither are present for a namespace package (it doesn't resolve to a file AND the package can exist on multiple search paths).

Fixes astral-sh/ty#375
Fixes astral-sh/ty#363

Test plan

Added regression tests. I verified that the imports in the linked issues resolve successfully.

Ecosystem review

• optuna: Correct, auto_generated is a namespace package
• rotki: Correct, the project has a packaging folder. It doesn't contain any python code but it isn't wrong for ty to pick it up as a namespace package.
• psycopg: All the code is in psycopg/psycopg (it's a src layout but the src folder is called psycopg). We weren't able to resolve any imports before. We now resolve psycopg as a namespace package which is correct from a module resolution point. The solution here is to change how we detect src.root or that they change their configuration to set src.root
• rclip: Is a namespace package and we are now able to resolve the imports
• dd-trace-py: benchmarks is a namespace package and we're now able to resolve imports. They also use other namespace packages that now resolve correctly
• scipy: They use submodules for some of the modules in _lib but they point to the project root and not the package folder? We now recognize them as namespace packages.

Unsure:

schema_salad:

The relevant lines are:

import ruamel.yaml from ruamel.yaml.comments import CommentedBase, CommentedMap, CommentedSeq

def _add_lc_filename(r: ruamel.yaml.comments.CommentedBase, source: AnyStr) -> None:

ruamel is a namespace package, ruamel.yaml is a regular package.

Before: We inferred Unknown for ruamel and yaml
Now: We infer ruamel.yaml but it seems we don't resolve the comments.py module?

I think this is unrelated to my changes.

cwltool: Same as schema_salad