| msg158281 - (view) |
Author: Stefan Krah (skrah) *  |
Date: 2012-04-14 20:45 |
| In the build --without-threads, catching an ImportError in support.py fails. Fedora buildbot: http://www.python.org/dev/buildbot/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/1986/steps/test/logs/stdio ./python ./Tools/scripts/run_tests.py -j 1 -u all -W --timeout=3600 Traceback (most recent call last): File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/runpy.py", line 160, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/runpy.py", line 73, in _run_code exec(code, run_globals) File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/__main__.py", line 1, in from test import regrtest, support File "<frozen importlib._bootstrap>", line 1038, in _handle_fromlist File "<frozen importlib._bootstrap>", line 977, in _find_and_load File "<frozen importlib._bootstrap>", line 561, in load_module File "<frozen importlib._bootstrap>", line 218, in module_for_loader_wrapper File "<frozen importlib._bootstrap>", line 446, in _load_module File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/regrtest.py", line 243, in from test import support File "<frozen importlib._bootstrap>", line 1038, in _handle_fromlist File "<frozen importlib._bootstrap>", line 977, in _find_and_load File "<frozen importlib._bootstrap>", line 561, in load_module File "<frozen importlib._bootstrap>", line 218, in module_for_loader_wrapper File "<frozen importlib._bootstrap>", line 446, in _load_module File "/home/buildbot/buildarea/3.x.krah-fedora/build/Lib/test/support.py", line 34, in import multiprocessing.process SystemError: NULL result without error in PyObject_Call [110893 refs] make: *** [buildbottest] Error 255 |
|
|
| msg158401 - (view) |
Author: Stefan Krah (skrah) * |
Date: 2012-04-16 06:41 |
| The SystemError has changed to a KeyError: http://www.python.org/dev/buildbot/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/2004/steps/test/logs/stdio At least that makes it easy to spot the location in import.c. The shortest way to reproduce the error is to compile --without-threads, then: import multiprocessing import multiprocessing.process |
|
|
| msg158472 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2012-04-16 15:44 |
| Just to clarify the failure for the bug history, somehow multiprocessing is not ending up in sys.modules as expected. It changed from a SystemError to a KeyError because I started to properly check a PyDict_GetItem() return value instead of blindly assuming something was going to be in sys.modules. |
|
|
| msg159500 - (view) |
Author: Stefan Krah (skrah) * |
Date: 2012-04-27 21:56 |
| This issue is now apparently causing a segfault: (gdb) r ./Tools/scripts/run_tests.py -j 1 -u all -W --timeout=3600 Starting program: /home/stefan/pydev/cpython/python ./Tools/scripts/run_tests.py -j 1 -u all -W --timeout=3600 [...] Program received signal SIGSEGV, Segmentation fault. 0x000000000041b038 in PyObject_Repr (v=<unknown at remote 0x7ffff43b7660>) at Objects/object.c:377 377 if (Py_TYPE(v)->tp_repr == NULL) (gdb) l [...] #0 0x000000000041b038 in PyObject_Repr (v=<unknown at remote 0x7ffff43b7660>) at Objects/object.c:377 #1 0x0000000000460db0 in PyUnicode_FromFormatV (format=0x63e3e8 "%R not in sys.modules as expected", vargs= 0x7fffffff1ae0) at Objects/unicodeobject.c:2609 #2 0x00000000004e1be2 in PyErr_Format (exception=<type at remote 0x8e68c0>, format= 0x63e3e8 "%R not in sys.modules as expected") at Python/errors.c:697 #3 0x00000000004ed53f in PyImport_ImportModuleLevelObject (name='multiprocessing.process |
|
|
| msg160086 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-05-06 16:09 |
| Here is a patch. The __import__ function's crazy API never ceases to amaze me. |
|
|
| msg160087 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-05-06 16:12 |
| Oops, there was a duplicate test. |
|
|
| msg160090 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2012-05-06 16:23 |
| Only two comments, otherwise LGTM (and I can't believe the solution was to go back through the import system just to pull out the cached module; the things we would change if we were doing this from scratch). One, you have some "XXX False" markers in the tests. Should those get deleted or replaced with something? Two, in your first test (at least) you only test what is in sys.modules once instead of after each attempted import. I would repeat the test after each import. |
|
|
| msg160092 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-05-06 16:31 |
| > One, you have some "XXX False" markers in the tests. Should those get > deleted or replaced with something? Well, I don't know what to replace them with. I would have expected pkg.module to end up in sys.modules, but as mentioned in the comments the relative import line actually fails with an ImportError. There's probably some logic behind that, but that's beyond me. > Two, in your first test (at least) you only test what is in > sys.modules once instead of after each attempted import. I would > repeat the test after each import. Ok, will do. |
|
|
| msg160094 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-05-06 16:35 |
| > > One, you have some "XXX False" markers in the tests. Should those get > > deleted or replaced with something? > > Well, I don't know what to replace them with. I would have expected > pkg.module to end up in sys.modules, but as mentioned in the comments > the relative import line actually fails with an ImportError. There's > probably some logic behind that, but that's beyond me. I should mention that the same happens with 2.7 and 3.2, so it's not a regression. It's just that I don't understand the logic :-) |
|
|
| msg160105 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2012-05-06 19:24 |
| So I was going to try to figure out the logic, so I manually created the test files to start debugging, but I didn't get the ImportError but instead the 1/0 error for the relative import. Maybe it's specific to lack of threads or the change you made? I mean if that's how it has always worked then I'm not arguing that it's wrong, just that it's a weird side-effect: >>> import pkg Traceback (most recent call last): File "", line 1, in File "<frozen importlib._bootstrap>", line 977, in _find_and_load File "<frozen importlib._bootstrap>", line 596, in load_module File "<frozen importlib._bootstrap>", line 262, in module_for_loader_wrapper File "<frozen importlib._bootstrap>", line 484, in _load_module File "./pkg/__init__.py", line 3, in 1/0 ZeroDivisionError: division by zero [70552 refs] >>> import sys [70554 refs] >>> sys.modules['pkg'] Traceback (most recent call last): File "", line 1, in KeyError: 'pkg' [70462 refs] >>> sys.modules['pkg.module'] <module 'pkg.module' from './pkg/module.py'> [70465 refs] >>> with open('pkg/__init__.py') as file: print(file.read()) ... from . import module #import pkg.module 1/0 |
|
|
| msg160106 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-05-06 19:42 |
| > So I was going to try to figure out the logic, so I manually created > the test files to start debugging, but I didn't get the ImportError > but instead the 1/0 error for the relative import. Maybe it's specific > to lack of threads or the change you made? I mean if that's how it has > always worked then I'm not arguing that it's wrong, just that it's a > weird side-effect: The first "import pkg" gets you a ImportError but the second one should get you a ZeroDivisionError (yes, weird). |
|
|
| msg160164 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2012-05-07 19:44 |
| New changeset d6324941b739 by Antoine Pitrou in branch 'default': Issue #14583: Fix importlib bug when a package's __init__.py would first import one of its modules then raise an error. http://hg.python.org/cpython/rev/d6324941b739 |
|
|
| msg160167 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-05-07 19:48 |
| Should be fixed now. The buildbot has been able to launch the test suite. |
|
|