[Python-Dev] Meta-reflections (original) (raw)

Moore, Paul Paul.Moore@atosorigin.com
Thu, 21 Feb 2002 13:19:47 -0000

From: Kevin Jacobs [mailto:jacobs@penguin.theopalgroup.com]

On Thu, 21 Feb 2002, Moore, Paul wrote: > > I agree, but that "official" support has clear limitations. > > I'm not sure what you mean.

When you request dir(object), there is a fairly significant amount of work done. [...] I know that we are talking about Python, and performance is not of paramount importance.

Hmm. I tend to favour "do it right, then do it fast". If there's a performance hit on dir(), why can't it be made faster? If nothing else, as a part of the core, dir() has the right to access dict and slots. So there's no a priori reason why dir() should be slower than any user-coded way of doing the same.

Of course, we really want vars() here, as we're otherwise doing work in dir() to get entries that we then throw away. But that's the only issue. Get vars() to work, and if it's too slow, you can argue that it's a bug because "I can get the same results by using the following code, which is faster".

I'm just not looking forward to 25% slowdowns in pickling (number pulled out of hat) and I'm sure the Zope guys aren't either...

There's bound to be some slowdown, from the (new) need to find slots as well as dict-based attributes. I'm happy if you want it minimised. But that's a new point you've raised, which I don't have the expertise to comment on.

That is not the bug -- if for no other reason, the standard library is free to use implementation specific knowledge. Getting obj.dict is a really slick and efficient way to reflect on all normal instance variables.

I'm not sure I agree here - it's better if the standard library uses interfaces which are available to the user. And if pickling can be made fast, why shouldn't the machinery that makes this possible be made available to the end user?

You could say that this argues in favour of making dict and slots part of the "official" reflection API. My view is that it argues for making the "official" API (which I'm assuming will be vars() for now) efficient enough that people don't need to use disct and slots. Encapsulation is good.

I don't see how filling slots with default values is compatible with the premise that we want slots to act as close to normal instance attributes as possible.

Fair enough. I offered that as one option. Clearly you prefer the other (that what's in dir() and/or slots cannot be guaranteed not to raise AttributeError). I'm happy either way, not having a vested interest in the issue.

> Why not just change the line stuff = object.dict to > > stuff = [a for a in dir(object) if hasattr(object,a) and not > callable(getattr(object,a))]

Um, because its wrong?

Sorry - it was an off-the-top-of-the-head suggestion. But it made my real point, which was that you can do it with dir().

stuff = dict([ (a,getattr(object,a)) for a in vars(object) if hasattr(object,a)])

Note that it does an unnecessary getattr, hasattr, memory allocation and incurs loop overhead on every dict attribute, but otherwise it should work once vars is fixed.

Efficiency again. I'd have to bow to your greater experience here. Although with pickling, doesn't I/O usually outweigh any performance cost?

> 3. Document dict as legacy usage, not slots-aware

Agree, though dict should still be a valid way of accessing all non-slot instance attributes. Too much legacy code would break if this were not so.

That's what I meant. Document it as the historical way of getting at instance attributes. Still available, but code which uses it will not support slots. After all, if you pass classes using slots into code which uses dict, things will go wrong. That's just another sort of breakage. Nobody's arguing that dict should go away. Except possibly from the documentation :-)

Calling base class setattr? I'm not sure what you mean?

It's in the part of the descrintro document I pointed you at. Traditional implementations of setattr used assignment to self.dict['attr'] to avoid infinite recursion. The "new way" discussed in the descrintro document is to call the base class setattr.

By your logic, people don't have any business reading dict, but they do.

They don't have any business any more. An important distinction. (And it's not anywhere near as black and white as that comment implies - I know that).

Imagine what would happen if we didn't expose dict in Python 2.3?

Nothing at all, if we provide an alternative. Except for backward compatibility issues, which there's a well-documented deprecation process to address. Of course, nobody is proposing the removal of dict. All I'm suggesting is that we document its limitations, point out better ways, and leave it at that.

Paul.