Hexchat (fork of XChat IRC Client) switched with version 2.9.6 to Python 3.3 for their Python plugins. Hexchat loads plugins in a similar fashion like the attached C file (not entirely sure I used the C-API right though). For every plugin a new interpreter is started and Py_EndInterpreter is used to unload the plugin. When using pickle in such a python plugin, it raises the Exception (in the attached example): Traceback (most recent call last): File "", line 7, in TypeError: attribute of type 'NoneType' is not callable when trying to pickle an object (using pickle.dump or pickle.dumps). The Exception happens on the line where pickle.dumps is called, though pickle.dumps is not None. It happens: - with python3.3.2 (I also tested it with python3.4.0a2 since I happened to have it installed, same issue). No issue with python2.7. I did not test it with 3.x versions prior to 3.3.2 - when trying to pickle a user defined class. Python objects like dictionaries and lists work fine. - only with the second (and any additional) interpreter. - when destroying the interpreter and starting a new one, in that order. When two interpreters are started and execute the code before any of them is destroyed, it works fine. The full output of the attached file when executed is btw: First output: Pickle dumps: b'\x80\x03c__main__\nSomeClass\nq\x00)\x81q\x01}q\x02X\x08\x00\x00\x00some_varq\x03K{sb.' Second output: Pickle dumps: Traceback (most recent call last): File "", line 7, in TypeError: attribute of type 'NoneType' is not callable
The NoneType that fails is in typeobject.c: _PyObject_CallMethodId(copyreg, &PyId__slotnames, "O", cls); The error here is that copyreg comes from a cached reference to the module, stored in a static variable (cached_copyreg_module). When the interpreter shuts down the first time, all the module members are set to None... but the reference is kept forever. This variable should be reset, and reloaded (with a fresh module from the new interpreter) the next time it's used. I added a call to _PyType_Fini() in Py_EndInterpreter, this fixes the given example. Two sets of questions though: - There are many of these _Fini functions, called in Py_Finalize. Which ones should we call in Py_EndInterpreter? and why? Maybe this PyType_Fini() is not a _Fini, but should be renamed PyType_EndInterpreter? - one copyreg for multiple interpreters... this looks wrong: each interpreter has its own list of modules, but copyreg.__globals__ belongs to only one... A good solution would be to cache the copyreg module in the PyInterpreterState (it already has codec_search_cache). Or use PyImport_GetModuleDict()
> one copyreg for multiple interpreters... this looks wrong: each interpreter has its own list of modules, but copyreg.__globals__ belongs to only one... > A good solution would be to cache the copyreg module in the PyInterpreterState (it already has codec_search_cache). Yes, this is a reasonable solution.
