[Python-3000] Draft pre-PEP: function annotations (original) (raw)

Talin talin at acm.org
Sat Aug 12 00:39:56 CEST 2006

Josiah Carlson wrote:

"Phillip J. Eby" <pje at telecommunity.com> wrote:

At 06:10 AM 8/11/2006 -0700, Talin <talin at acm.org> wrote:

Or to put it another way: If you create a tool, and you assume that tool will only be used in certain specific ways, but you fail to enforce that limitation, then your assumption will be dead wrong. The idea that there will only be a few type annotation providers who will all nicely cooperate with one another is just as naive as I was in the SysEx debacle. Are you saying that function annotations are a bad idea because we won't be able to pickle them? That is not what I got out of the message at all. If not, your entire argument seems specious. Actually, even if that is your argument, it's specious, since all that's needed to support pickling is to support pickling. All that's needed to support printing is to support printing (via str), and so on. I think you misunderstood Talin. While it was a pain for him to work his way through implementing all of the loading/etc. protocols, I believe his point was that if we allow any and all arbitrary metadata to be placed on arguments to and from functions, then invariably there will be multiple methods of doing as much. That isn't a problem unto itself, but when there ends up being multiple metadata formats, with multiple interpretations of them, and a user decides that they want to combine the functionality of two metadata formats, they may be stuck due to incompatibilities, etc. I think that it can be fixed by defining a standard mechanism for 'metadata chaining', one involving tuples and/or dictionaries. Say, for example, we have the following function definition: def foo(argn:meta=dflt): ... Since meta can take on the value of a Python expression (executed during compile-time), a tuple-based chaining would work like so: @chainmetadatatuple(metafcn1, metafcn2) def foo(argn:(meta1, meta2)=dflt): ... And a dictionary-based chaining would work like so: @chainmetadatadict(m1=metafcn1, m2=metafcn2) def foo(argn:{'m1'=meta1, 'm2'=meta2}=dflt): ... The reason to include the dict-based option is to allow for annotations to be optional. This method may or may not be good. But, if we don't define a standard method for metadata to be combined from multiple protocols, etc., then we could end up with incompatabilities. However, if we do define a standard chaining mechanism, then it can be used, and presumably we shouldn't run into problems relating to incompatible annotation, etc. - Josiah

Josiah is essentially correct in his interpretation of my views. I really don't understand what Phillip is talking about here.

Say I want to annotate a specific argument with two pieces of information, a type and a docstring. I have two metadata interpreters, one which uses the type information to restrict the kinds of arguments that can be passed in, and another which uses the docstring to enhance the generated documentation.

Now, lets say that these two metadata interpreters were written by two people, who are not in communication with each other. Each one decides that they would like to "play nice" with other competing metadata.

So Author A, who wrote the annotation decorator that looks for docstrings, decides that not only will he accept docstring annotations, but if the annotation is a tuple, then he will search that tuple for any docstrings, skipping over any annotations that he doesn't understand. (Although how he is supposed to manage that is unclear - since there could also be other annotations that are simple text strings as well.)

Author B, who wrote the type-enforcement module, also wants to play nice with others, but since he doesn't know A, comes up with a different solution. His idea is to create a system in which annotations automatically chain each other - so that each annotation has a "next" attribute referring to the next annotation.

So programmer C, who wants to incorporate both A and B's work into his program, has a dilemma - each has a sharing mechanism, but the sharing mechanisms are different and incompatible. So he is unable to apply both A-type and B-type metadata to any given signature.

What happens next is that C complains to both A and B (and in the process introducing them to each other.) A and B exchange emails, and reach the conclusion that B will modify his library to confirm to the sharing mechanism of A.

What this means is that A and B have created a defacto standard. Anyone who wants to interoperate with A and B have to write their interpreter to conform to the sharing mechanism defined by A and B.

But it also means that anyone outside of the ABC clique will not know about A&B's sharing convention, which means that their metadata interpreter will not be able to interoperate with A&B-style metadata. So in essence, A&B have now "captured" the space of annotations - that is, anyone who conforms to the A&B protocol can combine their annotations together; Anyone outside that group is excluded from interoperating.

Finally, lets say that A&B eventually become well-known enough that their sharing convention becomes the defacto standard. Any metadata that wants to interoperate with other metadata-interpretation libraries will have to follow the A&B convention. Any metadata library that chooses to use a different convention will be at a severe disadvantage, since they won't be able to be used together with other metadata interpreters.

What this means is that, despite the statements that annotations have no defined format or meaning, the fact is that they now do: The defacto A&B sharing convention. The sharing convention tells metadata interpreters how to distinguish between metadata that they can interpret, and how to skip over other metadata.

So in other words, since the original author of the annotation system failed to provide a convention for multiple annotations, they force the community to fill in the parts of the standard that they left out.

-- Talin

More information about the Python-3000 mailing list