[Python-Dev] Problem between deallocation of modules and func_globals (original) (raw)

M.-A. Lemburg mal at egenix.com
Sat Jan 20 12:31:45 CET 2007

On 2007-01-20 00:01, Brett Cannon wrote:

On 1/19/07, M.-A. Lemburg <mal at egenix.com> wrote:

On 2007-01-19 22:33, Brett Cannon wrote:

That's a typical error situation you get in del methods at the time the interpreter is shut down.

Yeah, but in this case this is at the end of PyInitialize() for the stuff I am doing to the interpreter. =) Is that in some error branch of PyInitialize() ? Otherwise I don't see how the modules could get garbage-collected. Nope, it's code I am adding to clean out sys.modules of stuff the user didn't import themselves; it's for security reasons.

I'm not sure whether that's really going to increase security: unloading of modules usually isn't safe and you cannot be sure that it's possible to reinitialize a C module once it has been loaded in the process. For Python modules this is often possible, but there still may be side-effects of the import that you cannot easily undo.

Perhaps you should just move those modules out to a different dictionary and keep track of it in the import mechanism, so that while you can't access the module directly via sys.modules, the import mechanism still knows that it has been loaded and reinserts it into sys.modules if it gets imported again.

I think that you get more security by explicitly limiting which modules and packages you allow to be imported in the first place and restricting what can be done with sys.path and sys.modules.

I'm not exactly sure which global state you are referring to. The aliase map, the cache used by the search function ?

encodings.cache . Note that the search function registry is a global managed in the thread state (it's not stored in any module). Right, but that is not the issue. If you have deleted the reference to the encodings module from sys.modules it then sets encodings.cache to None. After the deletion, if you try to encode/decode a unicode string you can an AttributeError about how encodings.cache does not have a 'get' method since it is now None instead of a dict. The function is fine and still runs, it's just that the global state it depends on is no longer the way it assume it should be. While I could add some tricks to have the cache dictionary stay alive even after the globals were set to None, I doubt that this will really fix the problem. The encoding package relies on the import mechanism, the codecs module and the codecs builtin module. Any of these could fail to work depending on the order in which the modules get GCed. There's a reason why things in PyFinalize() are as carefully ordered :-) Perhaps we need to apply some reordering to the steps in PyInitialize() ?! Nah, I just need to not delete the modules. =)

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Jan 20 2007)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Python-Dev mailing list