| msg177707 - (view) |
Author: anatoly techtonik (techtonik) |
Date: 2012-12-18 20:35 |
| Weird bug. Attached are two files: wow.py and wy.py When wow.py is executed - it fails with single ImportError. But when wy.py is executed (which is a simple "import wow" statement) it fails with one ImportError and TypeError exceptions. |
|
|
| msg177708 - (view) |
Author: anatoly techtonik (techtonik) |
Date: 2012-12-18 20:36 |
| ...one ImportError and *two* TypeError exceptions. |
|
|
| msg177709 - (view) |
Author: anatoly techtonik (techtonik) |
Date: 2012-12-18 20:39 |
| >py wy.py Traceback (most recent call last): File "wy.py", line 1, in import wow File "E:\scons\wow.py", line 12, in import fail ImportError: No module named fail Error in atexit._run_exitfuncs: Traceback (most recent call last): File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "E:\scons\wow.py", line 8, in _clean cleanlist = [ c for c in _Cleanup if c ] TypeError: 'NoneType' object is not iterable Error in sys.exitfunc: Traceback (most recent call last): File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "E:\scons\wow.py", line 8, in _clean cleanlist = [ c for c in _Cleanup if c ] TypeError: 'NoneType' object is not iterable |
|
|
| msg177710 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2012-12-18 20:48 |
| It's because _Cleanup is None (as all global variables) when atexit registered functions executed. Don't use globals in such code. |
|
|
| msg177716 - (view) |
Author: anatoly techtonik (techtonik) |
Date: 2012-12-18 21:19 |
| But why _Cleanup is not None when wow.py is executed standalone? |
|
|
| msg177724 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2012-12-19 00:10 |
| When you run wy.py the wow module gets partially imported, and then garbage collected because it fails to import successfully. The destructor for the module replaces values in the module's __dict__ with None. So when the cleanup function runs you get the unexpected error. When you run wow.py directly, wow (i.e. the main module) will not be garbage collected, so _Cleanup is never replaced by None. |
|
|
| msg177768 - (view) |
Author: anatoly techtonik (techtonik) |
Date: 2012-12-19 17:56 |
| This is not repeatable in Python 3. Is it possible to fix it for Python 2 as well? Is it possible to postpone registration until the import is finished successfully? Or at least give atexit handler a chance to run before global variable stack is purged? Or destroy atexit handler if its namespace is purged? |
|
|
| msg177773 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2012-12-19 19:38 |
| Infortunately, the behaviour is the same in 2.7 and 3.2, and only changes with 3.3, which means that this is probably related to the new implementation of the import system. This won't be backported. [It would be interesting to understand why there is a difference, btw... is a reference to the module held somewhere?] A possible workaround for such module-level calls atexit.register() is to ensure that used globals are captured in the function namespace, a bit like some __del__ methods: def _clean(_Cleanup=_Cleanup): .... |
|
|
| msg177777 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2012-12-19 20:23 |
| OK, found the difference between 3.2 and 3.3: The ImportError exception is still alive when atexit handlers are called, with its __traceback__, and all local variables in Python frames. And since 3.3 the import system is built with Python functions... More exactly, I could find the incomplete 'wow' module in sys.last_value.__traceback__.tb_next.tb_frame.f_back.f_back.f_back.f_locals['module'] But all this is not really related to the current issue. Things should be better in the future, when modules are cleared with true garbage collection. |
|
|
| msg177785 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2012-12-19 22:13 |
| > Things should be better in the future, when modules are cleared with true > garbage collection. When is this future of which you speak? I am not sure whether it would affect performance, but a weakrefable subclass of dict could be used for module dicts. Then the module destructor could just save the module's dict in a WeakValueDictionary keyed by the id (assuming we are not yet shutting down). At shutdown the saved module dicts could be purged by replacing all values with None. Or maybe something similar is possible without using a dict subclass. |
|
|
| msg177790 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2012-12-19 22:59 |
| See for the shutdown procedure and modules cleanup. |
|
|
| msg177791 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2012-12-19 23:06 |
| > See for the shutdown procedure and modules cleanup. I am aware of that issue, but the original patch is 9 years old. Which is why I ask if/when it will actually happen. |
|
|
| msg177802 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2012-12-20 08:17 |
| I don't know, you should probably ask there. One blocker is to make builtin and extension modules participate in GC, search "pep3121" to see the many intermediate issues. |
|
|
| msg177858 - (view) |
Author: Richard Oudkerk (sbt) * |
Date: 2012-12-20 20:00 |
| Perhaps the simplest thing would be to stop doing anything special when a module is garbage collected: the garbage collector can take care of any orphaned ref-cycles involving the module dict. Then at shutdown the remaining modules in sys.modules could have their dicts "purged" in the old way. This would be orthogonal to . In fact Armin's original post says that this is a change worth investigating, though his patch does not do it. |
|
|
| msg387053 - (view) |
Author: Irit Katriel (iritkatriel) * |
Date: 2021-02-15 20:17 |
| Since this was fixed since 3.3, is there a reason why this issue is still open? I don't get the error anymore: C:\Users\User\src\cpython-dev>python.bat wow.py Running Release|x64 interpreter... Traceback (most recent call last): File "C:\Users\User\src\cpython-dev\wow.py", line 12, in import fail ModuleNotFoundError: No module named 'fail' C:\Users\User\src\cpython-dev>python.bat wy.py Running Release |
x64 interpreter... Traceback (most recent call last): File "C:\Users\User\src\cpython-dev\wy.py", line 1, in import wow File "C:\Users\User\src\cpython-dev\wow.py", line 12, in import fail ModuleNotFoundError: No module named 'fail' |
|