[Python-Dev] need help with frozen module/marshal/gc issue involving sub-interpreters for importlib bootstrapping (original) (raw)

Guido van Rossum guido at python.org
Mon Feb 6 17:05:24 CET 2012

Usually this means that you're not doing an INCREF in a place where you should, and the object is kept alive by something else. Do you know which object it is? That might really help... Possibly deleting the last subinterpreter makes the refcount of that object go to zero. Of course it could also be that you're doing a DECREF you shouldn't be doing... But the identity of the object seems key in any case.

--Guido

On Mon, Feb 6, 2012 at 6:57 AM, Brett Cannon <brett at python.org> wrote:

So my grand quest for bootstrapping importlib into CPython is damn close to coming to fruition; I have one nasty bug blocking my way and I can't figure out what could be causing it. I'm hoping someone here will either know the solution off the top of their head or will have the time to have a quick look to see if they can figure it out as my brain is mush at this point.

First, the bug tracking all of this is http://bugs.python.org/issue2377and the repo where I have been doing my work is ssh:// hg at hg.python.org/sandbox/bcannon/#bootstrapimportlib (change as needed if you want an HTTPS checkout). Everything works fine as long as you don't use sub-interpreters via testcapi (sans some test failures based on some assumptions which can easily be fixed; the bug I'm talking about is the only real showstopper at this point). Here is the issue: if you run testcapi the code triggers an assertion of ``testsubinterps (main.TestPendingCalls) ... Assertion failed: (gc->gc.gcrefs != 0), function visitdecref, file Modules/gcmodule.c, line 327.``. If you run the test under gdb you will discover that the assertion is related to ref counts when collecting for a generation (basically the ref updating is hitting 0 when it shouldn't). Now the odd thing is that this is happening while importing frozen module code (something I didn't touch) which is calling marshal (something else I didn't touch) and while it is in the middle of unmarshaling the frozen module code it is triggering the assertion. Does anyone have any idea what is going on? Am I possibly doing something stupid with refcounts which is only manifesting when using sub-interpreters? All relevant code for bootstrapping is contained in Python/pythonrun.c:importinit() (with a little tweaking in the io module to delay importing the os module and making import.c always use import instead of using the C code). I'm storing the import function in the PyInterpreterState to keep separate state from the other interpreters (i.e. separate sys modules so as to use the proper sys.modules, etc.). But as I said, this all works in a single interpreter view of the world (the entire test suite doesn't trigger a nasty error like this). Thanks for any help people can provide me on this now 5 year quest to get this work finished. -Brett

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20120206/01ceeaa9/attachment.html>

More information about the Python-Dev mailing list