Issue 17481: inspect.getfullargspec should use signature (original) (raw)
Created on 2013-03-19 17:31 by michael.foord, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (31)
Author: Michael Foord (michael.foord) * 
Date: 2013-03-19 17:31
inspect.getfullargspec (and potentially getargspec?) could use function.signature if set. This allows functions that lie about their signature with the new introspection tool (Signature) to still work with older code.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-11 22:24
Please consider the attached patch (getargsspec_01.patch).
It modifies 'getargspec' and 'getfullargspec' to use the 'inspect.signature' API. The entire test suite passes just fine.
This also will address issue #16490.
I can also update the docs, if it's necessary.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-17 22:46
Can somebody review the patch? I'd be cool if this lands in 3.4.
Author: Terry J. Reedy (terry.reedy) *
Date: 2014-01-18 20:26
While I plan to change Idle code to use signature directly, instead of getfullargspec, I agree that changing inspect also, and in 3.4, is a good idea. It may, however, affect tests other than Idle's, if there are any other stdlib consumers. I will try to look at the patch after changing Idle.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2014-01-19 03:54
I upgraded the description to a "should". Argument Clinic and other changes in Python 3.4 greatly improve introspection support for various aspects of the runtime and standard library (for example, issue 20223 will handle the new functools.partialmethod support). By also making those enhancements available via getfullargpsec, we should significantly increase the amount of existing code which benefits automatically for those enhancements, as it will no longer require explicit migration to the new PEP 362 APIs at the application/library level.
Reviewing the patch now.
Author: Larry Hastings (larry) *
Date: 2014-01-19 03:59
If we modify inspect.getfullargspec, shouldn't we modify inspect.getargspec too? "deprecated" doesn't mean "unsupported", it means "not recommended for new code, please stop using it".
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-19 04:28
Larry, getargspec uses getfullargspec, so it's covered.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2014-01-19 04:36
getargspec() is just a thin wrapper around getfullargspec(), so we get that for free.
Similarly, getcallargs() is also implicitly updated to full PEP 362 support by changing the implementation of getfullargspec().
The other two standard library consumers appear to be IDLE (as Terry noted above) and xmlrpc.server.ServerHTMLDoc.docroutine (the self-documenting XML-RPC server API, which sadly appears to be lacking test cases).
Reviewing those cases (especially the XML-RPC one) have convinced me we need a backwards compatibility fallback in the updated getfullargspec implementation. Specifically, if the new API throws ValueError (because it can't find a way to build a coherent signature object), we should fall back to the old approach. Otherwise we run the risk of introducing unexpected exceptions into introspection code.
That is, the new call to signature in getfullargspec should look something like:
try:
sig = signature(func)
except ValueError:
if not hasattr(func, "__code__"):
raise # The legacy fallback doesn't support this input type
# The callable signature appears to be incoherent, so we fall
# back to the legacy implementation to preserve compatibility
args, varargs, kwonlyargs, varkw = _getfullargs(func.__code__)
return FullArgSpec(args, varargs, varkw, func.__defaults__,
kwonlyargs, func.__kwdefaults__, func.__annotations__)I suspect this may actually be a better solution for IDLE rather than updating it to call inspect.signature directly (since IDLE probably wants the more graceful fallback to the old behaviour).
The way I would document all this in What's New is to update the current argument clinic section to also explain that we've taken advantage of PEP 362 to improve introspection in multiple areas, including for code that is using the introspection APIs that predate PEP 362.
Author: Terry J. Reedy (terry.reedy) *
Date: 2014-01-19 10:33
The relevant code in CallTips.py is argspec = "" if hasattr(ob, 'call'): ... if isinstance(fob, (types.FunctionType, types.MethodType)): argspec = inspect.formatargspec(*inspect.getfullargspec(fob))
So I want to broaden the second condition (or remove it), but if it returns something .formatargspec cannot digest, the outer call will have to be conditioned. My impression is that str(signature object) returns pretty much what .formatargspec does, but I have not experimented yet.
My #20122 patch moves currently commented out tests in CallTips.py that use the above to test_calltips.py. I will commit as soon as I test on 3.4. Until then, getfullargspec is not being used in the Idle test suite.
Author: Larry Hastings (larry) *
Date: 2014-01-19 10:44
I just want to mention, while we're all thinking about this stuff: I plan to enhance the Signature object to reflect "optional groups". Right now Signature objects cannot express parameters that are optional but don't have a published default. (e.g. a string parameter for a builtin having a default of NULL). A lot of these are being converted using optional groups, and I want to make it at least possible for user code to introspect those functions and understand that those are optional parameters.
My original plan was to add a "group" member, containing an arbitrary identifier of the "group" this parameter belongs to. I'm not sure that's sufficient; I may also need a "parent_group" parameter, or something like that, to represent the group that this group is nested in. I'm still thinking about it. But I'm hoping to get to this in the next two or three days.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2014-01-19 12:15
Sounds reasonable - we'll need that to convert range() and slice() anyway (I totally failed at finding time to look at the builtins this weekend, but I'd still like to handle those before the 3rd beta).
An alternative would be to actually add "ParameterGroup" as a new class that contains parameters (assuming tuple isn't sufficient), so the structure of the parameters more directly matches the underlying signature.
Author: Yury Selivanov (Yury.Selivanov) *
Date: 2014-01-19 17:16
Larry,
I saw your message on the tracker regarding adding support for parameters groups to the signature object. Would you mind if I join the discussion with my ideas of how this feature might be implemented?
Yury
On Sunday, January 19, 2014 at 5:44 AM, Larry Hastings wrote:
Larry Hastings added the comment:
I just want to mention, while we're all thinking about this stuff: I plan to enhance the Signature object to reflect "optional groups". Right now Signature objects cannot express parameters that are optional but don't have a published default. (e.g. a string parameter for a builtin having a default of NULL). A lot of these are being converted using optional groups, and I want to make it at least possible for user code to introspect those functions and understand that those are optional parameters.
My original plan was to add a "group" member, containing an arbitrary identifier of the "group" this parameter belongs to. I'm not sure that's sufficient; I may also need a "parent_group" parameter, or something like that, to represent the group that this group is nested in. I'm still thinking about it. But I'm hoping to get to this in the next two or three days.
Python tracker <report@bugs.python.org (mailto:report@bugs.python.org)> <http://bugs.python.org/issue17481>
Author: Terry J. Reedy (terry.reedy) *
Date: 2014-01-19 20:07
Yuri, I am sure your ideas for enhancing signature objects would be welcome. Either a new issue or a thread on pydev or python-ideas lists would be best.
When responding by email, please snip the quotation and footer, except possibly a line of the quoted message.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-19 20:11
Terry,
Thanks.
When responding by email, please snip the quotation and footer, except possibly a line of the quoted message.
My email client played an evil (and a bit embarrassing) trick with me, showing Larry's name without an actual email address, which was "report@bugs.python.org". Won't happen again.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-19 22:18
Otherwise we run the risk of introducing unexpected exceptions into introspection code.
That's a good catch. I'll make a new patch, keeping the old implementation of getfullargsspec intact, and falling back to it if no signature can be found.
Author: Larry Hastings (larry) *
Date: 2014-01-19 22:22
Yury: fire away, either here or in a new issue as is best. (Sorry, brain mildly fried, not sure what would be best issue-tracker hygiene.)
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-19 22:56
Otherwise we run the risk of introducing unexpected exceptions into introspection code.
That's a good catch. I'll make a new patch, keeping the old implementation of getfullargsspec intact, and falling back to it if no signature can be found.
Nick, while I was working on the second patch (writing a new unittest for it specifically), I realized, that it's not that easy to make the old version of "getfullargsspec" to spit out any exception that it doesn't currently do with the proposed 'getargsspec_01.patch'.
See, the old "getfullargsspec" does the following:
Check if the passed object is a function, with 'inspect.isfunction'. If not - throw a TypeError. That behaviour is duplicated in the patch, so we are safe here.
Call on the object's code '_getfullargs', which validates that the passed code object is a valid code object, and simply returns its attributes rearranged a bit.
Now, to have any exception in (2), we need: either a broken code object, or something that is an instance of "types.FunctionType" (hence, defined in python) but doesn't have the "code" attribute. And that's kind of hard to achieve.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-20 02:45
Yury: fire away, either here or in a new issue as is best. (Sorry, brain mildly fried, not sure what would be best issue-tracker hygiene.)
Larry, I ended up with a relatively big core dump of my thoughts, so I decided to post it on python-dev. Let's discuss it there, and we can create an issue here once we decide something.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2014-01-20 12:10
Ah, I had indeed missed the fact that getfullargspec() was still calling isfunction(). In that case, is the patch currently actually buying us anything much beyond handling signature attributes? Most of the new types that inspect.signature() supports will still fail that preliminary check, so code will need to use the new API explicitly in order to benefit from it.
By contrast, if we remove the check, then there's a wider range of exceptions that may be thrown, but also a much wider variety of inputs supported.
Instead of keeping the check, we could just unconditionally convert exceptions from the signature call to a TypeError in order to maintain compatibility with the old external behaviour.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-20 16:02
Instead of keeping the check, we could just unconditionally convert exceptions from the signature call to a TypeError in order to maintain compatibility with the old external behaviour.
Agreed. See the new patch (getargsspec_02.patch)
Unfortunately, we have to keep the old 'ismethod' check in place for backwards compatibility purposes.
Larry,
The attached patch contains one failing unit-test: 'inspect.signature' returns 'None' for '_testapi.docstring_no_signature'. It should instead raise a ValueError.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-20 16:57
Larry,
I created a separate issue for that: http://bugs.python.org/issue20313
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-23 18:07
Larry, Nick,
So what's the resolution on this one? Do I have a green light?
Author: Larry Hastings (larry) *
Date: 2014-01-23 23:56
My solution for pydoc was to call isroutine() instead of isfunction(), and yes handle it throwing an exception.
(I just checked, and I wasn't catching TypeError, only ValueError. I'll fix that in my next patch for #20189.)
Author: Alyssa Coghlan (ncoghlan) *
Date: 2014-01-24 14:03
Another case of "don't land it until Larry has dealt with the builtins".
The patch itself looks fine, though :)
Author: Larry Hastings (larry) *
Date: 2014-01-24 14:25
The patch from #20189 has landed.
Author: Larry Hastings (larry) *
Date: 2014-01-25 03:56
There's a major difference between getfullargspec/getargspec and inspect.signature: getfullargspec shows you the "self" parameter for bound methods, and inspect.signature does not.
class C: ... def foo(self, a): pass ... c = C()
import inspect str(inspect.signature(c.foo)) '(a)' inspect.getfullargspec(c.foo) FullArgSpec(args=['self', 'a'], varargs=None, varkw=None, defaults=None, kwonlyargs=[], kwonlydefaults=None, annotations={}) inspect.getargspec(c.foo) ArgSpec(args=['self', 'a'], varargs=None, keywords=None, defaults=None)
This is why help() (currently) shows bound parameters for methods implemented in Python, but doesn't show them for methods implemented in C. pydoc uses inspect.getfullargspec for pure Python functions and inspect.signature for builtins.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-27 19:34
There's a major difference between getfullargspec/getargspec and inspect.signature: getfullargspec shows you the "self" parameter for bound methods, and inspect.signature does not.
Larry, yes, that's correct. The attached patch simulates this behaviour, with:
if ismethod(func):
func = func.__func__I'm attaching 'getargsspec_03.patch', as the previous one (02) was a bit crippled.
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-28 18:02
Larry and Nick,
Please review the final patch (getargsspec_04.patch). I'd like to commit it tomorrow.
This one is the last one (hopefully), and it supports builtin methods properly -- i.e. works around 'self' parameter correctly. To do that, I check if the 'text_signature' starts with "($", etc.
Author: Roundup Robot (python-dev) 
Date: 2014-01-29 16:45
New changeset 6d1e8162e855 by Yury Selivanov in branch 'default': inspect.getfullargspec: Use inspect.signature API behind the scenes #17481 http://hg.python.org/cpython/rev/6d1e8162e855
Author: Yury Selivanov (yselivanov) *
Date: 2014-01-29 16:46
Committed. Thanks for the reviews!
Author: Roundup Robot (python-dev)
Date: 2014-01-29 16:54
New changeset 0fa2750c7241 by Yury Selivanov in branch 'default': inspect.test.getfullargspec: Add a unittest to ensure correct annotations http://hg.python.org/cpython/rev/0fa2750c7241
History
Date
User
Action
Args
2022-04-11 14:57:43
admin
set
github: 61683
2014-01-29 20:55:18
yselivanov
link
2014-01-29 16:54:26
python-dev
set
messages: +
2014-01-29 16:46:22
yselivanov
set
status: open -> closed
resolution: fixed
2014-01-29 16:46:15
yselivanov
set
messages: +
2014-01-29 16:45:48
python-dev
set
nosy: + python-dev
messages: +
2014-01-28 18:22:04
yselivanov
set
keywords: + needs review
2014-01-28 18:02:10
yselivanov
set
files: + getargsspec_04.patch
assignee: yselivanov
messages: +
stage: patch review
2014-01-27 19:34:29
yselivanov
set
files: + getargsspec_03.patch
messages: +
2014-01-25 03:56:05
larry
set
messages: +
2014-01-24 14:25:39
larry
set
messages: +
2014-01-24 14:03:29
ncoghlan
set
dependencies: + inspect.Signature doesn't recognize all builtin types
messages: +
2014-01-23 23:56:37
larry
set
messages: +
2014-01-23 18:07:46
yselivanov
set
messages: +
2014-01-20 16:57:15
yselivanov
set
messages: +
2014-01-20 16:02:06
yselivanov
set
files: + getargsspec_02.patch
messages: +
2014-01-20 12:10:19
ncoghlan
set
messages: +
2014-01-20 02:45:50
yselivanov
set
messages: +
2014-01-19 22:56:42
yselivanov
set
messages: +
2014-01-19 22:22:52
larry
set
messages: +
2014-01-19 22🔞39
yselivanov
set
messages: +
2014-01-19 20:11:36
yselivanov
set
messages: +
2014-01-19 20:07:36
terry.reedy
set
messages: +
2014-01-19 17:16:57
Yury.Selivanov
set
nosy: + Yury.Selivanov
messages: +
2014-01-19 12:15:03
ncoghlan
set
messages: +
2014-01-19 10:44:43
larry
set
messages: +
2014-01-19 10:33:46
terry.reedy
set
messages: +
2014-01-19 04:36:06
ncoghlan
set
messages: +
2014-01-19 04:28:28
yselivanov
set
messages: +
2014-01-19 03:59:02
larry
set
messages: +
2014-01-19 03:54:37
ncoghlan
set
messages: +
title: inspect.getfullargspec could use __signature__ -> inspect.getfullargspec should use __signature__
2014-01-18 20:26:24
terry.reedy
set
messages: +
2014-01-18 14:35:13
yselivanov
set
nosy: + terry.reedy
2014-01-17 22:46:20
yselivanov
set
nosy: + larry
messages: +
2014-01-11 23:38:06
Claudiu.Popa
set
nosy: + Claudiu.Popa
2014-01-11 22:24:56
yselivanov
set
files: + getargsspec_01.patch
nosy: + yselivanov
messages: +
keywords: + patch
2013-03-19 17:31:32
michael.foord
create