[Python-Dev] Meta-reflections (original) (raw)

Kevin Jacobs jacobs@penguin.theopalgroup.com
Fri, 22 Feb 2002 10:05:09 -0500 (EST)

On Fri, 22 Feb 2002, Samuele Pedroni wrote:

I was thinking along the line of the C equiv of this: [...code snipped...]

[ An updated version of a comment to SF issue: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=520644&group_id=5470 ]

Samuele's sltattr.py is an interesting approach, though I am not entirely sure it is sufficient to address all of the problems with slots. Here is a mostly complete list of smaller changes that are somewhat orthogonal to how we address accesses to dict:

  1. Flatten slot lists: Change obj.class.slots to return an immutable list of all slot descriptors in the object (including all those of base classes). The motivation for this is similar in spirit to storing a flattened mro.

    The advantages of this change are:

    a) allows for fast and explicit object reflection that correctly finds all dict attributes, all slot attributes.

    b) allows reflection implementations (like vars(object) and pickle) to treat dict and slot attrs differently if we choose not to proxy dict. This has several advantages, as explained in change #2. Also importantly, this way it is not possible to "lose" descriptors permanently by deleting them from obj.class.dict.

  2. Update reflection API even if we do not choose to proxy dict: Alter vars(object) to return a dictionary of all attributes, including both the contents of the non-proxied dict and the valid attributes that result from iterating over slots and evaluating the descriptors. The details of how this is best implemented depend on how we wish to define the behavior of modifying the resulting dictionary. It could be either:

      a) explicitly immutable, which involves creating proxy objects
  b) mutable, which involves copying
  c) undefined, which means implicitly immutable

    Aside from the questions over the nature of the return type, this implementation (coupled with #1) has distinct advantages. Specifically the native object.dict has a very natural internal representation that pairs attribute names directly with values. In contrast, a fair amount of additional work is needed to extract the slots that store values and create a dictionary of their names and values. Other implementations will require a great deal more work since they would have to traverse though base classes to collecting slot descriptors.

  3. Flatten slot inheritance: Update the new-style object inheritance mechanism to re-use slots of the same name, rather than creating a new slot and hiding the old. This makes the inheritance semantics of slots equivalent to those of normal instance attributes and avoids introducing an ad-hoc and obscure method of data hiding.

  4. Update standard library to use new reflection API (and make them robust to properies at the same time) if we choose not to proxy dict. Virtually all of the changes are simple and involve updating these constructs:

      a) obj.__dict__
  b) obj.__dict__[blah]
  c) obj.__dict__[blah] = x

    (What these will become depends on other factors, including the context and semantics of vars(obj).)

    Here is a fairly complete list of Python 2.2 modules that will need to be updated: copy, copy_reg, inspect, pickle, pydoc, cPickle, Bastion, codeop, dis, doctest, gettext, ihooks, imputil, knee, pdb, profile, rexec, rlcompleter, tempfile, unittest, xmllib, xmlrpclib

5) (NB: potentially controversial and not required) We could alter the
   descriptor protocol to make slots (and properties) more transparent
   when the values they reference do not exist.  Here is an example to
   illustrate this:

       class A(object):
         foo = 1

       class B(A):
         __slots__ = ('foo',)

       b = B()
       print b.foo
       > 1 or AttributeError?

    Currently an AttributeError is raised.  However, it is a fairly easy
    change to make AttributeErrors signal that attribute resolution is
    to continue until either a valid descriptor is evaluated, an
    instance-attribute is found, or until the resolution fails after
    search the meta-type, the type, and the instance dictionary.

I am prepared to submit patches to address each of these issues. However, I do want feedback beforehand, so that I do not waste time implementing something that will never be accepted.

Regards, -Kevin

-- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com