[Python-3000] PEP 3124 - more commentary (original) (raw)

Guido van Rossum guido at python.org
Tue May 15 00:43:50 CEST 2007

On 5/14/07, Phillip J. Eby <pje at telecommunity.com> wrote:

At 12:47 PM 5/14/2007 -0700, Guido van Rossum wrote: >> >I realize that @overload is only a shorthand for @when(function). But >> >I'd much rather not have @overload at all -- the frame inspection >> >makes it really hard for me to explain carefully what happens without >> >just giving the code that uses sys.getframe(); and this makes it >> >difficult to reason about code using @overload. >> >>This is why in the very earliest GF discussions here, I proposed a >>'defop expr(...)' syntax, as it would eliminate the need for any >>getframe hackery. > >But that would completely kill your "but it's all pure Python code so >it's harmless and portable" argument.

Uh, wha? You lost me completely there. A 'defop' syntax simply eliminates the need to name the target function twice (once in the decorator, and again in the 'def'). I don't get what that has to do with stuff being harmless or portable or any of that. Are you perhaps conflating this with the issue of marking functions as overloadable? These are independent ideas, AFAICT. >It seems that you're really not interested at all in compromising to >accept mandatory marking of the base overloadable function. Uh, wha? I already agreed to that a couple of weeks ago: http://mail.python.org/pipermail/python-3000/2007-May/007205.html I just haven't updated the PEP yet -- any more than I've updated it with anything else that's been in these ongoing threads, like the :nextmethod annotation or splitting the PEP.

Ah, sorry. The way this misunderstanding probably originated was that I read your "this is why I originally proposed defop, to avoid getframe hackery" as a maintaining the current need for getframe, instead of a historical fact.

>>Anyway, with this, it could also be placed as a keyword >>argument. The main reason for putting it in the first position is >>performance. Allowing it to be anywhere, however, would let the >>choice of where be a matter of style. > >Right. What's the performance issue with the first argument?

Chaining using the first argument can be implemented using a bound method object, which gets performance bonuses from the C eval loop that partial() objects don't. (Of course, when RuleDispatch was written, partial() objects didn't exist, anyway.)

Sounds like premature optimization to me. We can find a way to do it fast later; let's first make it right.

>>However, since we're going to have to have some way for 'super' to >>know the class a function is defined in, ISTM that the same magic >>should be reusable for the first-argument rule. > >Perhaps. Though super only needs to know it once the method is being >called, while your decorator (presumably) needs to know when the >method is being defined, i.e. before the class object is constructed.

Not really; at some point the class object has to be assigned and stored somewhere for super to use, so if same process of "assigning" can be used to actually perform the registration, we're good to go.

True. So are you working with Tim Delaney on this? Otherwise he may propose a simpler mechanism that won't allow this re-use of the mechanism.

>Also, the similarities between next-method and super are overwhelming. >It would be great if you could work with Tim Delaney on a mechanism >underlying all three issues, or at least two of the three.

I'm not sure I follow you. Do you mean, something like using :super as the annotation instead of nextmethod, or are you just talking about the implementation mechanics?

super is going to be a keyword with magic properties. Wouldn't it be great if instead of

@when(...) def flatten(x: Mapping, nm: next_method): ... nm(x)

we could write

@when(...) def flatten(x: Mapping): ... super.flatten(x) # or super(x)

or some other permutation of super? Or do you see the need to call both next-method and super from the same code?

>> >Forgive me if this is mentioned in the PEP, but what happens with >> >keyword args? Can I invoke an overloaded function with (some) keyword >> >args, assuming they match the argument names given in the default >> >implementation? >> >>Yes. That's done with code generation; PEAK-Rules uses direct >>bytecode generation, but a sourcecode-based generation is also >>possible and would be used for the PEP implementation (it was also >>used in RuleDispatch). > >There's currently no discussion of this.

Well, actually there's this bit: """The use of BytecodeAssembler can be replaced using an "exec" or "compile" workaround, given a reasonable effort. (It would be easier to do this if the funcclosure attribute of function objects was writable.)""" But the closure bit is irrelevant if we're using @overloadable.

Thanks.

>Without a good understanding >of the implementation I cannot accept the PEP.

The mechanism is exec'ing of a string containing a function definition. The original function's signature is obtained using inspect.getargspec(), and the string is exec'd to obtain a new function whose signature matches, but whose body contains the generic function lookup code.

Do note that e.g. in IronPython (and maybe also in Jython?) exec/eval/compile are 10-50x slower (relative to the rest of the system) than in CPython.

It does look like a clever approach though.

In practice, the actual function definition has to be nested, so that argument defaults can be passed in without needing to convert them to strings, and so that the needed lookup tables can be seen via closure variables. A string template would look something like:

_def makethefunction(_defaults, lookup): def funcname(funcname(funcname(acceptsignature): _return lookup($typetuple)($callsignature) return $funcname The $typetuple bit would expand to something like: type(firstargname), type(secondargname), ... And $acceptsignature would expand to the original function's _signature, with default values replaced by "defaults[0]", _"defaults[1]", etc. in order to make the resulting function have the same default values. The function that would be returned from @overloadable would be the result of calling "makethefunction", passing in the original _function's funcdefaults and an appropriate value for lookup. A similar approach is used in RuleDispatch currently.

>> >Also, can we overload different-length signatures (like in C++ or >> >Java)? This is very common in those languages; while Python typically >> >uses default argument values, there are use cases that don't easily >> >fit in that pattern (e.g. the signature of range()). >> >>I see a couple different possibilities for this. Could you give an >>example of how you'd like it to work? > >In the simplest case (no default argument values) overloading two-arg >functions and three-arg functions with the same name should act as if >there were two completely separate functions, except for the base >(default) function. Example: > >@overloadable >def range(start:int, stop:int, step:int): > ... # implement xrange > >@range.overload >def range(x): return range(0, x, 1) > >@range.overload >def range(x, y): return range(x, y, 1) Hm. I'll need to give some thought to that, but it seems to me that it's sort of like having None defaults for the missing arguments, and then treating the missing-argument versions as requiring type(None) for those arguments. Except that we'd need something besides None, and that the overloads would need wrappers that drop the extra arguments. It certainly seems possible, anyway. I'm not sure I like it, though.

C++ and Java users use it all the time though.

It's not obvious from the first function's signature that you can call it with fewer arguments, or what that would mean. For example, shouldn't the later signatures be "range(stop)" and "range(start,stop)"? Hm.

I don't know if the arg names for overloadings must match those of the default function or not -- is that specified by your PEP?

My own trivially simple overloading code (sandbox/overload, and now also added as an experiment to sandbox/abc, with slightly different terminology and using issubclass exclusively, as you recommended over a year ago :-) has no problem with this. Of course it only handles positional arguments and completely ignores argument names except as keys into the annotations dict.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-3000 mailing list