[Python-Dev] Rough idea for adding introspection information for builtins (original) (raw)

Stefan Behnel stefan_ml at behnel.de
Tue Mar 19 07:08:48 CET 2013

Larry Hastings, 19.03.2013 05:45:

The original impetus for Argument Clinic was adding introspection information for builtins [...] On to the representation. Consider the function

def foo(arg, b=3, *, kwonly='a'): pass [...] 4. Store a string that looks like the Python declaration of the signature, and parse it (Nick's suggestion). For foo above, this would be "(arg,b=3,*,kwonly='a')".

I had already noted that this would be generally useful, specifically for Cython, so I'm all for going this route. No need to invent something new here.

Length: 23 bytes.

I can't see why the size would matter in any way.

Of those, Nick's suggestion seems best. It's slightly bigger than the specialized bytecode format, but it's human-readable (and human-writable!), and it'd be the easiest to implement.

Plus, if it becomes the format how C level signatures are expressed anyway, it wouldn't require any additional build time preprocessing.

My first idea for implementation: add a "def x" to the front and ": pass" to the end

Why not require it to be there already? Maybe more like

def foo(arg, b=3, *, kwonly='a'):
     ...

(i.e. using Ellipsis instead of pass, so that it's clear that it's not an empty function but one the implementation of which is hidden)

then run it through ast.parse. Iterate over the tree, converting parameters into inspect.Parameters and handling the return annotation if present. Default values and annotations would be turned into values by ast.evalliteral. (It wouldn't surprise me if there's a cleaner way to do it than the fake function definition; I'm not familiar with the ast module.)

IMHO, if there is no straight forward way currently to convert a function header from a code blob into a Signature object in Python code, preferably using the ast module (either explicitly or implicitly through inspect.py), then that's a bug.

We'd want one more mild hack: the DSL will support positional parameters, and inspect.Signature supports positional parameters, so it'd be nice to render that information. But we can't represent that in Python syntax (or at least not yet!), so we can't let ast.parse see it. My suggestion: run it through ast.parse, and if it throws a SyntaxError see if the problem was a slash. If it was, remove the slash, reprocess through ast.parse, and remember that all parameters are positional-only (and barf if there are kwonly, args, or kwargs).

Is sounds simpler to me to just make it a Python syntax feature. Or at least an optional one, supported by the ast module with a dedicated compiler flag.

Stefan

More information about the Python-Dev mailing list