Challenges of Static Typing for Curried Functions in Python (original) (raw)

The issue arose while I was developing a utility library and attempted to use a static type system to annotate a curried function. I found this to be nearly impossible.

For example:

class File:
    path: PathLike
    ...
    def open(self, *args, **kw):
        return open(self.path, *args, **kw)

I cannot accurately describe the type of the File.open function because its types are strongly tied to the types of the built-in open function.

In fact, within static typing, I cannot easily and precisely obtain something like “the type of the first parameter of function a” or “the number of parameters of function a.”

When the target function is simple, everything seems fine. I can simply copy the types of each parameter and the return value from the target function and paste them into my own function. But I believe you can already see how ridiculous this is:

It is completely detrimental to maintainability and readability. It tightly couples my function with the external target function. If the target function changes, it would be catastrophic for the static type annotations in my own module.

In more realistic scenarios, most complex functions’ type systems also include overloads, variable keyword arguments, and variable positional arguments, which make things even more overwhelming. It feels like an almost impossible task.

Does anyone have any good ideas or suggestions?

MegaIng (Cornelius Krupp) April 23, 2025, 3:48pm 2

Calling this “curried function” is a confusing use of terminology.

There have been a few previous discussions:

• @extends / @copy_signature to add arguments to an overriden method of a derived class
• Taking the argument signature from a different function
• Extract kwargs types from a function signature

All in all, getting a good syntax for this working is difficult, especially since we don’t have function signature literals.

I think someone needs to do a lot of work of reading through all these (and probably more) discussions and make a proposal that can be implemented and discussed.

InSync (InSync) April 23, 2025, 4:20pm 3

JamesParrott (James Parrott) April 23, 2025, 6:26pm 4

If this isn’t a simplified example for explanatory purposes, then pathlib.Path already has a .open method.
At the risk of getting carried away, the official type annotation for open has 7 different cases (which the stubs for pathlib.Path.open pretty much duplicate).

dave-shawley (Dave Shawley) April 24, 2025, 10:40pm 5

I can agree that it is maddening that something as “simple” as extending a function causes a type checking conundrum. I’ve usually fallen back on ParamSpec when I could and (luckily) most of the time I’ve encountered this I was writing a decorator. I think that the best way to start out is to examine the conversations that have been had (see above) and come up with a proposal for what we want it to look and work like. Then we can start the discussion on how to make that work.

I really wanted a way to extract a ParamSpec from an existing function. This seems like the first step. Once you have a ParamSpec you can use Unpack and Concatenate to build your function’s signature. Something like the following maybe:

OpenParams = typing.copy_signature(open)

def my_open(
    custom_param: str, *args: OpenParams.args,
    **kwargs: OpenParams.kwargs
) -> OpenParams.return_annotation:
    print(custom_param)
    return open(*args, **kwargs)

Mind you that the above is complete fallacy. typing.copy_signature would be a static version of what inspect.signature does. Type checking tools would have to recognize it as producing a typing.ParamSpec that matches the function parameter.

So… would something like that satisfy the discussions that have already been had? If so, what will it take to get a proof-of-concept into pyright, mypy, or whatever? If not, what needs to change?

I don’t think that there is a disagreement that the functionality is desirable. I know that I have wanted it more than a few times. I think that it hasn’t risen up high enough for anyone to spend time designing, proposing, and then implementing a PoC.