[Python-Dev] Choosing a best practice solution for Python/extension modules (original) (raw)

Jean-Paul Calderone exarkun at divmod.com
Sat Feb 21 18:17:49 CET 2009

On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon <brett at python.org> wrote:

On Fri, Feb 20, 2009 at 12:53, Aahz <aahz at pythoncraft.com> wrote:

On Fri, Feb 20, 2009, Brett Cannon wrote: > On Fri, Feb 20, 2009 at 12:37, Brett Cannon <brett at python.org> wrote: >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach <_ _>> daniel at stutzbachenterprises.com> wrote: >>> >>> A slight change would make it work for modules where only key functions >>> have been rewritten. For example, pickle.py could read: >>> >>> from pypickle import * >>> try: from pickle import * >>> except ImportError: pass >> >> True, although that still suffers from the problem of overwriting things >> like name, file, etc. > > Actually, I take that back; the IMPORTSTAR opcode doesn't pull in anything > starting with an underscore. So while this alleviates the worry above, it > does mean that anything that gets rewritten needs to have a name that does > not lead with an underscore for this to work. Is that really an acceptable > compromise for a simple solution like this?

Doesn't all control this? If you define it, yes. But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to pypickle and not _pickle. So if you have something like:: # pypickle def A(): return B() def B(): return -13 # pickle def B(): return 42 # pickle from pypickle import * try: from pickle import * except ImportError: pass If you import pickle and call pickle.A() you will get -13 which is not what you are after.

If pickle and _pypickle are both Python modules, and _pypickle.A is intended to be used all the time, regardless of whether _pickle is available, then there's not really any reason to implement A in _pypickle. Just implement it in pickle. Then import whatever optionally fast thing it depends on from _pickle, if possible, and fall-back to the less fast thing in _pypickle otherwise.

This is really the same as any other high-level/low-level library split. It doesn't matter that in this case, one low-level implementation is provided as an extension module. Importing the low-level APIs from another module and then using them to implement high-level APIs is a pretty common, simple, well-understood technique which is quite applicable here.

Jean-Paul

More information about the Python-Dev mailing list