[Python-Dev] Meta-reflections (original) (raw)

Moore, Paul Paul.Moore@atosorigin.com
Thu, 21 Feb 2002 09:58:36 -0000

From: Kevin Jacobs [mailto:jacobs@penguin.theopalgroup.com]

Having a flat namespace (i.e., no hidden slots), and having all 'reachable' slots in a single list are really two separate issues. Right now, we have this situation:

class Foo(object): slots = ('a','b') class Bar(Foo): slots = ('c','d') bar=Bar() print bar.slots > ('c', 'd') print bar.class.base.slots > ('a', 'b') I content that bar.slots should be: > ('a', 'b', 'c', 'd') I think somewhere along the line I may have mixed up which 'flatness' I was> talking about.

Um. Be aware that I'm not 100% sure about the "no hidden slots" point. I only support it on the basis of acting the same as attributes. But I'm not sure about it for attributes, either... (although I doubt it will ever change).

As for your contention over slots, I don't agree. I don't have a strong view, but I feel that slots is really only meant as a way of defining the slots (and as such, may be replaced later by better syntax). Think of it as write-only. Modifying it after the class is defined, or reading it, aren't really well defined (and don't need to be). If slot definition had been spelt "slot a" instead of "slots = ['a']", you wouldn't necessarily expect to have a readable attribute containing the list of slots...

> The official, and supported, way is to use dir().

I agree, but that "official" support has clear limitations.

I'm not sure what you mean.

Right now, there are several examples in the Python standard library where we use obj.dict.keys() -- most significantly in pickle and cPickle.

But aren't we agreed that this is the source of a bug (that slots aren't picklable)?

There is also the vars(obj), which may be the better reflection method, though it currently doesn't know about slots.

Possibly you're right. This could easily be raised as a feature request. And possibly even as a bug (vars() should know about slots).

Naive, maybe, but saying "undocumented" is equivalent to "unsupported implementation detail" saves us from having to maintain backward compatibility and following the official Python deprecation process.

Equally, saying "it's not in the manual, so tough luck" is unreasonably harsh. We need to be reasonable about this.

> Do you have any reason why you would need to get a list of > only slots, or only dict-based attributes?

Yes. Dict-based attributes always have values, while slot-based attributes can be unset and raise AttributeErrors when trying to access them.

Hmm. I could argue this a couple of ways. Slots should contain None when unassigned (no AttributeErrors), or code should be (re-)written to cope with AttributeError from things in dir(). I wouldn't argue it as "slots can raise AttributeError, so we need to treat slots and dict-based attibutes separately, in 2 passes".

here is how I would handle pickling (excerpt from pickle.py):

try: getstate = object.getstate except AttributeError: stuff = object.dict # added to support slots if hasattr(object.slots): for slot in object.slots: if hasattr(object, slot): stuff[slot] = getattr(object, slot) else: stuff = getstate() keepalive(stuff, memo) save(stuff) write(BUILD)

Why not just change the line stuff = object.dict to

stuff = [a for a in dir(object) if hasattr(object,a) and not

callable(getattr(object,a))]

[The hasattr() gets rid of unbound slots - this may be another argument for unbound slots containing None, and the callable() gets rid of methods]. Then slots are covered, as well as any future non-dict-based attribute types.

If vars(obj) was fixed to include slots, like dir() was, then this could be reduced to "stuff = vars(object)" (modulo protection against AttributeError).

I'm not suggesting anything more incestuous and low-level than what is already done in many, many, many places. A larger, more-encompassing proposal is definitely welcome.

I'm not sure we need a larger proposal. A smaller one may work better. I'm arguing above for

  1. Make unbound slots return None rather than AttributeError
  2. Make vars() return slots as well as dict-based attributes
  3. Document dict as legacy usage, not slots-aware
  4. Fix bugs caused by use of the legacy dict approach
  5. Educate users in the new approaches which are slots-aware (dir/vars, calling base class setattr, etc)

(and maybe a sixth, don't make slots a reflection API - make it an implementation detail, a bit like dict is now viewed)

Well, I've not found resounding agreement on the first two of my three basic issues/bugs I've raised so far:

1) Flat slot namespaces: Objects should not hiding slots when inherited by classes implementing the same slot name.

You're right - I'm not in "resounding" agreement. I think it's probably better, for consistency with dict-based attributes, but I sort of wish it wasn't. (The fact that I've not hit problems because of the equivalent property of attributes means that I'm probably wrong, and attributes are fine as they are, though.)

2) Flat slot descriptions: object.slots being an immutable flat tuple of all slot names (including inherited ones), as opposed to being a potentially mutable sequence of only the slots defined by the most derived class.

This I disagree with. I think slots should be immutable. But I'm happy with "don't do that" as the way of implementing immutability, if that's what Guido prefers. I definitely don't think slots should return something different when you read it than what you assigned to it in the first place (which is what including inherited slots does). But I don't really think people have any business reading slots in any case (see my arguments above).

3) First class status for slot reflection: making slots picklable by default, returned by vars(object), and made part of other relevant reflection APIs and standard implementations.

I agree on this one.

Paul.