[ty] Create fresh copies of generic callable typevars by dcreager · Pull Request #24949 · astral-sh/ruff (original) (raw)

Merged

dcreager

merged 32 commits into

Jun 8, 2026

Conversation

A generic callable binds new typevars. If that callable is used more than once in a particular expression, we should create separate (aka "fresh") copies of those typevars, so that the inferred types for each do not conflict with each other.

This comes up in two situations:

• We might invoke a generic function recursively from inside of its body. In that case, the call site introduces fresh inferable copies of the function's typevars, which should not conflict with the existing non-inferable copies from the function signature.
• We might pass the same generic callable more than once to some other call. In this case, each occurrence should have distinct copies of its typevars, that do not need to unify with each other. This can occur whenever we compare a generic callable signature for assignability with some other signature.

This PR updates these two places (call binding and signature assignability checking) to introduce fresh copies of the callable's typevars when needed.

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 92.16%. The percentage of expected errors that received a diagnostic held steady at 87.31%. The number of fully passing files held steady at 92/134.

Memory usage report

Summary

Significant changes

Click to expand detailed breakdown

prefect

sphinx

trio

flake8

ecosystem-analyzer results

Raw diff:

Expression (https://github.com/cognitedata/Expression)

• expression/collections/maptree.py:176:36 error[invalid-argument-type] Argument to function try_find is incorrect: Expected Option[MapTreeLeaf[Key@try_find, Value@try_find]], found Option[MapTreeLeaf[Never, Never]]

xarray (https://github.com/pydata/xarray)

• xarray/backends/api.py:1363:42 error[invalid-argument-type] Argument to function _remove_path is incorrect: Expected NestedSequence[_FLike@_remove_path], found (_FLike@_remove_path & Top[list[Unknown]]) | (NestedSequence[_FLike@_remove_path] & Top[list[Unknown]])
• xarray/core/utils.py:356:35 error[invalid-argument-type] Argument to function flat_items is incorrect: Expected Mapping[str, dict[str, Divergent] | Unknown], found dict[str, Divergent] | (T@flat_items & Top[dict[Unknown, Unknown]])

• xarray/structure/combine.py:69:57 error[invalid-argument-type] Argument to function _infer_tile_ids_from_nested_list is incorrect: Expected NestedSequence[Unknown], found object

• xarray/structure/combine.py:69:57 error[invalid-argument-type] Argument to function _infer_tile_ids_from_nested_list is incorrect: Expected NestedSequence[T@_infer_tile_ids_from_nested_list], found object

Full report with detailed diff (timing results)

Merging this PR will degrade performance by 7.38%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 (👁 2) regressed benchmarks
✅ 65 untouched benchmarks
⏩ 60 skipped benchmarks1

Performance Changes

Comparing dcreager/alpha-renaming (dca5384) with main (b5fce66)

Footnotes

1. 60 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

jelle-openai added a commit to jelle-openai/ruff that referenced this pull request

May 27, 2026

This was referenced

May 28, 2026

All of the ecosystem changes are exposing existing feature gaps in new locations. Though the changed diagnostic in xarray does show the intended fix from this PR — we now see distinct uses of the typevar of this recursive function.

The reduced performance seems reasonable for the new (and necessary) work that's being performed. There are similar slowdowns in a handful of ecosystem projects, but most of them are in line with the previous timing numbers.

dcreager marked this pull request as ready for review

June 2, 2026 16:56

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a "shared delta"? I guess I might've considered this the "nonce" itself? Can we give it a dedicated type?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're adding a value to the existing nonce to make sure that the result is unique. So if we have

Callable[[T'0, T'1], T'2]

(where each T is the same typevar, but with different nonces), we need to bump everything by 3 to make sure that the result doesn't conflict with any of the existing nonces:

Callable[[T'3, T'4], T'5]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me from reading the code and the summary, though I again don't feel super well positioned to give substantive feedback. I think that's fine, just want to call it out.

Should "freshen" just be "copy" or something?

I'm a little saddened by the memory increase here, though I can also try to reduce that hit separately.

I think "freshen" is the terminology mypy uses for this too internally FWIW — don't know if that's something @dcreager was aware of before this PR or if he came to the same terminology independently 😆

If it's used elsewhere no objection from me for sure

Codex gave me this for reducing memory significantly: #25670

Codex gave me this for reducing memory significantly: #25670

Nice! Merged into here

I think "freshen" is the terminology mypy uses for this too internally FWIW — don't know if that's something @dcreager was aware of before this PR or if he came to the same terminology independently 😆

I didn't know about mypy using it, but it's a common term in the literature for this too!

dcreager deleted the dcreager/alpha-renaming branch

June 8, 2026 19:33

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

Labels

ty

Multi-file analysis & type inference