[Python-Dev] PySet API (original) (raw)

Barry Warsaw barry at python.org
Sun Mar 26 00:03:35 CET 2006

On Tue, 2006-03-21 at 21:31 -0500, Raymond Hettinger wrote:

[Barry] > Is it your intent to push for more use of the abstract API instead of > the concrete APIs for all of Python's C data structures? Current API > aside, are you advocating this approach for all new built-in types? > Would you argue that Python 3.0's C API be stripped of everything but > the abstract API and the bare essentials of the concrete API?

It's not up to me. Perhaps someone else can chime-in about the philosophy of how the C API is supposed to balance abstract and concrete APIs.

I think it's an important point to discuss, both for trying to resolve this impasse and for helping to direct future API designs, especially as we get into Python 3.0.

Maybe it will help you to understand why I want a richer concrete API. I work on an app that is deeply integrated with Python. It's hard to say whether we embed or extend -- it's a lot of both. We use Python data structures such as lists, dicts, and sets in many places as our fundamental tracking objects. So we know what we have, i.e. it's definitely a set here and another set there, and we want to merge one into the other. Or we know we have a set of foo's here and we need to iterate over them quickly (yes, in a tight loop) to count things or whatever.

So there's no question that a concrete API is very useful to us. And there's no questions that snaking through the abstract API causes us real debugging pain (a point which you mostly glossed over). We understand the gotchas about reference counting and the possibilities and implications about calling back into Python. Remember, we're all consenting adults here. I don't think we're unique here, as the rich concrete API of other fundamental Python objects attests to.

Your comments lead me to think that you aren't taking this important use case into account. You talk about duck typing, but I don't care about that here. I absolutely know I have a PySet, so why cause me pain to use it?

I know that the more one uses the abstract API, the more likely the code is going to be able to accept duck typed inputs. Also, most things that have tpslots have a corresponding abstract method instead of tons a concrete access points; hence, I would be supportive if you proposed a PyObjectClear(o) function (for calling tpclear slots when they exist and returning an error code when they don't).

I wouldn't object to that, but it wouldn't change my mind about PySet_Clear(). I'm not arguing against a rich abstract API, I'm arguing for having a richer concrete API too. And in this case, only slightly richer.

For setobject.c, if I still have a say in the matter, my strong preference is to keep the API minimal, expose fine-grained functions for efficiency, use PyNumber methods for direct access to operator style set operations, and use the abstract API for everything else.

I think this is a silly stance. You agree that PySet_Next() is easier to use than the iterator API. We will definitely not use the latter, and if your position stands, then we'll just have to hack Python to add it (or implement it in an auxiliary module). But I don't want to have to do that, so I really don't understand your reluctance to add three obviously useful functions.

Another point: these don't expose internal bits of the set implementation. Well, except for the opaque position pointer, but that's still enough data hiding for me because you're never supposed to /do/ anything with that variable except pass it right back to PySet_Next(). PySet_Clear() and PySet_Update() don't expose any implementation details -- that's the whole point!

P.S. One other thought: I don't want to crystalize the API in a way that precludes future development of the module. One possibility for the future is for updates to take multiple arguments such as s.update(t,u,v) causing three updates to be folded-in at once.

I don't see any way that my proposals preclude that. And besides, the three API calls I'm proposing are useful /today/.

But just so we all know what we're talking about, I've uploaded the patch to SourceForge:

http://sourceforge.net/tracker/index.php?func=detail&aid=1458476&group_id=5470&atid=305470

As with all good patches, there's (almost) more test code than implementation.

Cheers, -Barry

-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 309 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20060325/dc65d315/attachment.pgp

More information about the Python-Dev mailing list