(original) (raw)

On 01/20/2014 04:59 AM, Nick Coghlan wrote:

When I wrote that, I was thinking we had made  
inspect.Signature.\_\_repr\_\_ produce a nice string format, but then I  
noticed in the REPL today that we never got around to doing that - I  
think because we didn't know how to handle positional-only arguments,  
which already can't be expressed as Python syntax. (I haven't checked  
if we have an RFE filed anywhere)

I don't know what you had intended to do, but right now inspect.Signature inherits the standard repl from object. inspect.Signature.\_\_str\_\_ produces something that looks like a Python function signature, starting and ending with parentheses. (For those of you unfamiliar with inspect.Signature: A signature is agnostic about the name of the function. So it doesn't include the name.)

However, while I know you're keen to finally make introspection work  
for all C level callables in 3.4, even the ones with signatures that  
can't be expressed as Python function signatures, I'd like to strongly  
encourage you to hold off on that last part until Python 3.5.

If we hold off on all of this until 3.5, the signatures for most builtins will be wrong in 3.4, because most builtins take positional-only parameters. I had higher hopes for Python 3.4 than that. To be honest I'd rather not have the feature at all than have it be wrong most of the time.

I think it's fair to summarize your argument as "there could be monsters lurking in CPython with signatures that can't be expressed in PEP 457 syntax". To me this smacks of FUD. Let me open my kimono and tell you all the counter-examples we know of so far.

socket.sendto() has an optional group in the middle of required parameters. (This signature is from 1993.) PEP 457 could support this just by relaxing one requirement. I know what's needed here, but given that PEP 457 was such a dud I haven't bothered to update it. Regardless, Argument Clinic, and the syntax used for text signatures, could (and I expect will soon) support this. The inspect.Parameter.group proposal from my email last night supports this just fine.
itertools.repeat() has a parameter that behaves differently if it's passed by keyword vs passed by position. Guido already ruled that this signature must be changed so it is representable with Python syntax--this behavior is a "bug".
Many functions have default values that are not representable in Python, chiefly a NULL pointer. Guido has already ruled that these signatures should be changed so that they're representable in Python. The best approach is often accepting None, which is inconvenient for non-pointer arguments like integers. Right now Argument Clinic gives you no assistance in this area, but I plan to add explicit support making it easy (via "nullable ints").

In short, there's a clear trend: functions must have signatures representable in Python syntax, with the exception of optional groups which are a legacy feature we can't get rid of but won't support in Python syntax. Any functions whose signatures are not representable in Python syntax shall be tweaked until they are.

Any new monsters we discover lurking in CPython will be slain, not supported.

\-----

We could split the difference, and not add a feature to the inspect module to support optional groups. We could still support marking positional-only parameters, as inspect currently supports that. That would mean nearly all signatures for builtins would be correct.

Personally I'd rather go the extra distance and support optional groups too. There are important callables that can only be expressed with optional groups (range, type). Given the trend above, Parameter arguments with optional groups should be sufficient to express every signature available in Python. We've come this far... or, as the British say, in for a penny, in for a pound. Let's hash it out right now and get it done.

While the text string used to communicate between Argument Clinic and  
inspect.signature will be private, the representation on  
inspect.Signature objects will be a new \*public\* API. As the  
discussions between you, me and Yury show, I don't think there's an  
immediately obvious best answer of how to do that. Your suggestion of  
just adding the group numbers to the Parameter objects would \*work\*,  
but it's not very Pythonic - we have container types that support  
nesting,

Apparently you didn't read my proposal in the email you replied to. I didn't propose that "group" contain a number, I proposed it contain a ParameterGroup object that supports nesting.

We could take another approach, one you seem to be suggesting, where the nesting is outside the Parameter objects. In this alternate approach, the Signature.parameters array can contain either Parameter objects or OrderedDicts. The nested OrderedDicts themselves can contain either Parameter objects or more nested OrderedDicts. The API would specify that the nested OrderedDicts of parameters are optional en masse. This works fine too.

The chief difference between these proposals: if you ignore the complexity of optional groups, the failure mode with ".group" is that it kind of works except when it doesn't, whereas with having OrderedDicts in .parameters the failure mode is that your code blows up with missing attributes (like "couldn't find an attribute called name on this OrderedDict object"). That's probably a vote in favor of the nested OrderedDicts.

/arry