[Python-Dev] PEP 318: How I would implement decorators (original) (raw)

Jim Hugunin lists at hugunin.net
Tue Apr 6 18:09:37 EDT 2004

I've spent a couple days reading the decorators discussion and thinking about the issues. That's not a lot of time, but I still thought I'd share my opinions on implementing this feature. This all is in my (mostly) humble opinion.

When I started looking at PEP 318, I thought that the most important issue was deciding on the correct syntax. After a while I became much more worried about semantics.

One of the nice properties of C# and Java attributes is that they are declarative rather than transformative so there are no conflicts between different attributes placed on the same object. This is probably too strong of a restriction for Python; nevertheless, it's a valuable reference point.

Using the definition of returns from the PEP, I get the following behavior if I confuse its order with classmethod, i.e.

class C: [classmethod(m), returns(str)] def m(cls): return repr(cls)

C.m() --> TypeError: unbound method new_f() must be called with C instance as first argument (got nothing instead)

This is a hard message to understand, but at least it does occur at the first instance of calling the method. Even worse is where you forget to create an instance of a class or to call a function like returns, i.e.

class C: [returns] def m(self): return "hello"

c = C() print c.m() #there's no error for this call --> function new_f at 0x01278BB0>

This kind of error scares me a lot. It can be extremely hard to track down because it doesn't occur either at the point of declaration or the point of calling the method, but only at some later stage where the result from the method is used. I'm also concerned that this error is caused by the person using the attribute not the person writing it and that person may be less sophisticated and more easily confused.

A less critical, but important issue is the propagation of doc strings and other function attributes, i.e.

class C: [returns(str)] def m(self): "a friendly greeting" return "hello"

print C.m.doc --> None

This will really mess up programs that rely on doc strings or other function attributes.

After thinking about these issues, my first inclination was to sign up for Raymond Hettinger's NO on 318 campaign. Unfortunately, as my previous message showed there are a number of ways that I'd really like to use decorators in my Python code.

My next thought was to propose that Python's decorators be restricted to be declarative like C# and Java attributes. They could be used to add meta information to functions, but other meta-tools would be needed for any actual transformations. I'm still open to that idea, but I've come to believe that's not a very pythonic approach.

My current idea is that we should design this feature and any documentation around it to strongly encourage people to write safe, composable decorators. Specifically, it should be easier to write safe and composable decorators than to write bad ones. After that, we can rely on decorator developers to do the right thing.

  1. Use a "decorate" method instead of a call to avoid the worst problems like [returns]. This has the added benefit in my eyes of making it a little harder to write small decorators and thus encouraging people to think a little more before writing these meta-programs.

  2. Provide a small number of helper classes that make it easy to write safe and composable decorators. Here are my top two candidates:

class attribute: def init(self, **kwds): self.attrs = kwds

def decorate(self, decl):
    if not hasattr(decl, __attrs__):
        decl.__attrs__ = []
    decl.__attrs__.append(self)
    return decl

def __repr__(self): ...

class func_wrapper: """subclass this and implement a wrap function""" def decorate(self, func): assert iscallable(func) new_func = self.wrap(func) assert iscallable(new_func) self.copy_attributes(func, new_func) return new_func

def copy_attributes(self, old_func, new_func):
    ...

func_wrapper would be used for things like synchronized and memoize, and attribute would be used for most of the other examples in my previous message.

  1. Provide a clear set of guidelines for good decorators, i.e. a. decorators should not have side effects outside of the class/function b. decorators should return the same kind of thing they are given c. decorators should be commutative whenever possible (requires b)

There should be a consensus on whether or not decorators like 'generic' are a good or a bad idea. If they're a good idea then the decorate method should get an extra argument exposing the local namespace so that sys._getframe() won't be required. I'd be in favor of adding that argument to decorate.

These two decorators are a big problem because they don't return the same kind of thing as they are given. This means they always have to be the last decorator applied in any list. That kind of restriction is never a good thing.

Personally, I'd like to see these changed from returning a new object to instead setting an attribute on the existing function/method object. Then the method object could use this attribute to implement its get method correctly. This would make these decorators compose very nicely with all of the other decorators; however, it would be a fairly major change to how Python handles static and class methods and I'm doubtful that would fly.

I've come to see syntax as less crucial than I originally thought. In particular, the examples I've read on this list of using temporary variables showed me that even if the syntax is too confining there are tolerable solutions. Nevertheless, I'm still strongly in favor of putting the decorator list in front of the def/class.

Decorators/attributes are an exciting new technique that's gaining increasing use in different programming languages. They're important enough to be one of the key ideas that Java-1.5 is stealing from C# (turn-about is always fair play). The AOP community often has vehement debates not about the value of attributes, but about how they can be most effectively used.

Because decorators are still a relatively new idea, it's hard to know how much room they will need. Certainly if we're only interested in classmethod and staticmethod it's not much. However, I think it's important to give decorators as much room as possible to be prepared for the future.

I have some sympathy with the arguments that a bare prefix list is already legal Python syntax and is certainly found in existing code. I'm quite sure that using a bare list would break existing Python programs that have dangling lists left over as they evolved.

I like Sam's proposal to add a '*' in front of the list. I could also be persuaded to go along with any of the new grouping constructing, like <| |>; however, those all seem like too much of a change for the value. I'm fairly confident that the *[] form would allow for attributes to optionally be put on the same line as the def. This would be LL1, but perhaps the Python parser has other restrictions that I'm unaware of, i.e.

*[java_sig("public void meth(int a, int b)"), staticmethod] def meth(self, a, b): pass

or for simple cases

*[staticmethod] def meth(self, a, b): pass

-Jim

More information about the Python-Dev mailing list