Issue 24213: ProcessPoolExecutor().map() fails following an identical map() (original) (raw)
Attached file runs fine. Uncommenting line 19 makes python crash. Also, no crash happens if order of execution of map() and executor.map() is switched.
This problem seems to be related to numpy in some way, the code does not crash if the commented return of function user_square is substituted for numpy functions.
Using a mac with Yosemite 10.10.3. When program crashed I get window that says:
" Python quit unexpectedly.
Click Reopen to open the application again. Click Report to see more detailed information and send a report to Apple. "
This is the traceback:
Traceback (most recent call last): File "processExample.py", line 27, in main() File "processExample.py", line 22, in main result3 = list(executor.map(user_square, nums)) File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 549, in result_iterator yield future.result() File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 402, in result return self.__get_result() File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/_base.py", line 354, in __get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
I can reproduce a crash with your sample program using the current numpy wheel (1.9.2) and the python.org 3.4.3. The significant parts of the OS X crash dump:
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000110
VM Regions Near 0x110: --> __TEXT 0000000100000000-0000000100001000 [ 4K] r-x/rwx SM=COW /Library/Frameworks/Python.framework/Versions/3.4.3_10_6/Resources/Python.app/Contents/MacOS/Python
Application Specific Information: crashed on child side of fork pre-exec
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libdispatch.dylib 0x00007fff97347c13 dispatch_group_async + 533 1 libBLAS.dylib 0x00007fff96271a67 dsyrk_r + 565 2 libBLAS.dylib 0x00007fff96272183 APL_dsyrk + 370 3 libBLAS.dylib 0x00007fff96370696 cblas_dgemm + 1144 4 _dotblas.so 0x0000000104216775 dotblas_matrixproduct + 5733 5 org.python.python 0x00000001000e44ce PyEval_EvalFrameEx + 28894 6 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 7 org.python.python 0x000000010003e8ba function_call + 186 8 org.python.python 0x000000010000d3c8 PyObject_Call + 104 9 org.python.python 0x00000001000e0cb9 PyEval_EvalFrameEx + 14537 10 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 11 org.python.python 0x000000010003e8ba function_call + 186 12 org.python.python 0x000000010000d3c8 PyObject_Call + 104 13 org.python.python 0x00000001000e0cb9 PyEval_EvalFrameEx + 14537 14 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 15 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 16 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 17 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 18 org.python.python 0x000000010003e8ba function_call + 186 19 org.python.python 0x000000010000d3c8 PyObject_Call + 104 20 org.python.python 0x000000010002802c method_call + 140 21 org.python.python 0x000000010000d3c8 PyObject_Call + 104 22 org.python.python 0x000000010007b831 slot_tp_init + 81 23 org.python.python 0x0000000100072d14 type_call + 212 24 org.python.python 0x000000010000d3c8 PyObject_Call + 104 25 org.python.python 0x00000001000e1b09 PyEval_EvalFrameEx + 18201 26 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 27 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 28 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 29 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 30 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 31 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 32 org.python.python 0x000000010003e8ba function_call + 186 33 org.python.python 0x000000010000d3c8 PyObject_Call + 104 34 org.python.python 0x00000001000e0cb9 PyEval_EvalFrameEx + 14537 35 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 36 org.python.python 0x00000001000e4306 PyEval_EvalFrameEx + 28438 37 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 38 org.python.python 0x00000001000e4306 PyEval_EvalFrameEx + 28438 39 org.python.python 0x00000001000e4455 PyEval_EvalFrameEx + 28773 40 org.python.python 0x00000001000e5ced PyEval_EvalCodeEx + 2349 41 org.python.python 0x00000001000e5daf PyEval_EvalCode + 63 42 org.python.python 0x000000010011048e PyRun_FileExFlags + 206 43 org.python.python 0x000000010011083d PyRun_SimpleFileExFlags + 717 44 org.python.python 0x000000010012810e Py_Main + 3262 45 org.python.python 0x0000000100000e32 0x100000000 + 3634 46 org.python.python 0x0000000100000c84 0x100000000 + 3204
Thread 0 crashed with X86 Thread State (64-bit): rax: 0x0000000105001480 rbx: 0x0000000000000000 rcx: 0x0000000000000000 rdx: 0x0000000000000100 rdi: 0x00007fff7cd5fd40 rsi: 0x0000000000000000 rbp: 0x00007fff5bffbbd0 rsp: 0x00007fff5bffbba0 r8: 0x0000000105001400 r9: 0x0000000000000000 r10: 0x00000000000007d0 r11: 0xffff8001a446c4d8 r12: 0x00000001004682b0 r13: 0x0000000100437210 r14: 0x00007fff97341317 r15: 0x0000000000002000 rip: 0x00007fff97347c13 rfl: 0x0000000000010206 cr2: 0x0000000000000110
Logical CPU: 2 Error Code: 0x00000006 Trap Number: 14 [...] 0x104212000 - 0x104217ff7 +_dotblas.so (???) <15CB5D81-FD91-3005-A21C-A7D1432020C2> /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/numpy/core/_dotblas.so [...] 0x7fff96258000 - 0x7fff963e6fff libBLAS.dylib (1128) <497912C1-A98E-3281-BED7-E9C751552F61> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib [...]
So the crash is occurring as a result of a call from NumPy's _dotblas extension module to OS X's libBLAS. There's nothing obvious here that would indicate this is a problem in Python itself. Suggest you pursue this in the NumPy community and, if necessary, open an issue there: