[Python-Dev] Choosing a best practice solution for Python/extension modules (original) (raw)

Brett Cannon brett at python.org
Sat Feb 21 20:07:07 CET 2009

On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone <exarkun at divmod.com>wrote:

On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon <brett at python.org> wrote:

On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:

On Fri, Feb 20, 2009, Brett Cannon wrote: > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote: >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <_ _>> daniel at stutzbachenterprises.com> wrote: >>> >>> A slight change would make it work for modules where only key functions >>> have been rewritten. For example, pickle.py could read: >>> >>> from pypickle import * >>> try: from pickle import * >>> except ImportError: pass >> >> True, although that still suffers from the problem of overwriting things >> like name, file, etc. > > Actually, I take that back; the IMPORTSTAR opcode doesn't pull in anything > starting with an underscore. So while this alleviates the worry above, it > does mean that anything that gets rewritten needs to have a name that does > not lead with an underscore for this to work. Is that really an acceptable > compromise for a simple solution like this?

Doesn't all control this?

If you define it, yes. But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to pypickle and not pickle. So if you have something like:: # pypickle def A(): return B() def B(): return -13 # pickle def B(): return 42 # pickle from pypickle import * try: from pickle import * except ImportError: pass If you import pickle and call pickle.A() you will get -13 which is not what you are after. If pickle and pypickle are both Python modules, and pypickle.A is intended to be used all the time, regardless of whether pickle is available, then there's not really any reason to implement A in pypickle. Just implement it in pickle. Then import whatever optionally fast thing it depends on from pickle, if possible, and fall-back to the less fast thing in pypickle otherwise. This is really the same as any other high-level/low-level library split. It doesn't matter that in this case, one low-level implementation is provided as an extension module. Importing the low-level APIs from another module and then using them to implement high-level APIs is a pretty common, simple, well-understood technique which is quite applicable here.

But that doesn't provide a clear way, short of screwing with sys.modules, to get at just the pure Python implementation for testing when the extensions are also present. The key point in trying to figure this out is to facilitate testing since the standard library already uses the import * trick in a couple of places.

-Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20090221/cbdb2ce9/attachment.htm>

More information about the Python-Dev mailing list