[ty] Recognize submodules in self-referential imports by sharkdp · Pull Request #18005 · astral-sh/ruff (original) (raw)

added the ty

Multi-file analysis & type inference

label

May 10, 2025

This was referenced

May 10, 2025

sharkdp marked this pull request as ready for review

May 11, 2025 09:52

dcreager added a commit that referenced this pull request

May 13, 2025

…eep-dish

• origin/main: [ty] file is always a string inside a Python module (#18071) [ty] Recognize submodules in self-referential imports (#18005) [ty] Add a note to the diagnostic if a new builtin is used on an old Python version (#18068) [ty] Add tests for else branches of hasattr() narrowing (#18067) [ty] Improve diagnostics for assert_type and assert_never (#18050) [ty] contribution guide (#18061) [ty] Implement DataClassInstance protocol for dataclasses. (#18018) [ruff_python_ast] Fix redundant visitation of test expressions in elif clause statements (#18064)

Glyphack pushed a commit to Glyphack/ruff that referenced this pull request

May 21, 2025

Summary

Fix the lookup of submodules in cases where the parent module has a self-referential import like from parent import submodule. This allows us to infer proper types for many symbols where we previously inferred Never. This leads to many new false (and true) positives across the ecosystem because the fact that we previously inferred Never shadowed a lot of problems. For example, we inferred Never for os.path, which is why we now see a lot of new diagnostics related to os.path.abspath and similar.

import os

reveal_type(os.path)  # previously: Never, now: <module 'os.path'>

closes astral-sh/ty#261 closes astral-sh/ty#307

Ecosystem analysis

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃ Diagnostic ID                 ┃ Severity ┃ Removed ┃ Added ┃ Net Change ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│ call-non-callable             │ error    │       1 │     5 │         +4 │
│ call-possibly-unbound-method  │ warning  │       6 │    26 │        +20 │
│ invalid-argument-type         │ error    │      26 │    94 │        +68 │
│ invalid-assignment            │ error    │      18 │    46 │        +28 │
│ invalid-context-manager       │ error    │       9 │     4 │         -5 │
│ invalid-raise                 │ error    │       1 │     1 │          0 │
│ invalid-return-type           │ error    │       3 │    20 │        +17 │
│ invalid-super-argument        │ error    │       4 │     0 │         -4 │
│ invalid-type-form             │ error    │     573 │     0 │       -573 │
│ missing-argument              │ error    │       2 │    10 │         +8 │
│ no-matching-overload          │ error    │       0 │   715 │       +715 │
│ non-subscriptable             │ error    │       0 │    35 │        +35 │
│ not-iterable                  │ error    │       6 │     7 │         +1 │
│ possibly-unbound-attribute    │ warning  │      14 │    31 │        +17 │
│ possibly-unbound-import       │ warning  │      13 │     0 │        -13 │
│ possibly-unresolved-reference │ warning  │       0 │     8 │         +8 │
│ redundant-cast                │ warning  │       1 │     0 │         -1 │
│ too-many-positional-arguments │ error    │       2 │     0 │         -2 │
│ unknown-argument              │ error    │       2 │     0 │         -2 │
│ unresolved-attribute          │ error    │     583 │   304 │       -279 │
│ unresolved-import             │ error    │       0 │    96 │        +96 │
│ unsupported-operator          │ error    │       0 │    17 │        +17 │
│ unused-ignore-comment         │ warning  │      29 │     2 │        -27 │
├───────────────────────────────┼──────────┼─────────┼───────┼────────────┤
│ TOTAL                         │          │    1293 │  1421 │       +128 │
└───────────────────────────────┴──────────┴─────────┴───────┴────────────┘

Analysis complete. Found 23 unique diagnostic IDs.
Total diagnostics removed: 1293
Total diagnostics added: 1421
Net change: +128

• We see a lot of new errors (no-matching-overload) related to os.path.dirname and other os.path operations because we infer str | None for __file__, but many projects use something like os.path.dirname(__file__).
• We also see many new unresolved-attribute errors related to the fact that we now infer proper module types for some imports (e.g. import kornia.augmentation as K), but we don't allow implicit imports (e.g. accessing K.auto.operations without also importing K.auto). See astral-sh/ty#133.
• Many false positive invalid-type-form are removed because we now infer the correct type for some type expression instead of Never, which is not valid in a type annotation/expression context.

Test Plan

Added new Markdown tests

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})