[Python-Dev] Inconsistency of PyModule_AddObject() (original) (raw)

Stefan Krah stefan at bytereef.org
Thu Apr 28 11:29:11 EDT 2016

Serhiy Storchaka <storchaka gmail.com> writes:

2. Most code that use PyModuleAddObject() doesn't work as intended. Since the bahavior of PyModuleAddObject() contradicts the documentation and is contrintuitive, we can't blame authors in this.

I don't say this is a high-impacting bug, I even agree that there is no need to fix the second part in maintained releases. But this is a bug unless you propose different definition for a bug.

Why do you think that module authors don't know that? For _decimal, I was aware of the strange behavior. Yes, a single reference can "leak" on failure.

The problem is that we don't seem to have any common ground here.

Do you accept the following?

  1. PyModule_AddObject() can only fail if malloc() fails.
a) Normally (for small allocations) this is such a serious problem
   that the whole application fails anyway.

b) Say that you're lucky and the application continues.

     i) The import fails. In some cases ImportError is caught and
        a fallback is imported (example _pydecimal). In that case
        you leak an entire DSO and something small like a single
        context object. What is the practical difference between the
        two?

    ii) The import fails and there's no fallback. Usually the
        application stops, otherwise DSO+small leak again.

   iii) Retry the import (I have never seen this):

          while(1):
              try:
                  import leftpad
              except (ImportError, MemoryError):
                  continue
              break

        You could have a legitimate leak here, but see a).

Module initializations are intricate and boring. I suspect that if we promote wide changes across PyPI packages we'll see more additional segfaults than theoretically plugged memory leaks.

Stefan Krah

More information about the Python-Dev mailing list