| msg294473 - (view) |
Author: Peter Parente (Peter Parente) |
Date: 2017-05-25 12:21 |
| Working on this PR (https://github.com/maxpoint/spylon/pull/49), I encountered an unexpected SystemError in Python 3.6 on my Mac and on Travis Linux. Exception in thread Thread-4: Traceback (most recent call last): File "/Users/parente/miniconda3/envs/spylon-dev/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/Users/parente/miniconda3/envs/spylon-dev/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/Users/parente/projects/spylon/spylon/spark/progress.py", line 99, in _spark_progress_thread_worker td = datetime.datetime.now() - start_times[stage_id] SystemError: <built-in method __missing__ of collections.defaultdict object at 0x106013098> returned NULL without setting an error The exception only occurs in Python 3.6, not in Python 3.5 or 3.4: https://travis-ci.org/maxpoint/spylon/builds/235992988 The defaultdict is both created and used as a local variable in a threading.Thread run() function. It's never accessed outside that context (https://github.com/maxpoint/spylon/pull/49/commits/2bd47dc32f6129f5f6a4824be1eaed568351df11#diff-1ba10d1cd92a35a380442cb586e310e2R179) I didn't see a related issue on the tracker here nor did I see anything in the 3.6 changelog about a behavior change. |
|
|
| msg294477 - (view) |
Author: Peter Parente (Peter Parente) |
Date: 2017-05-25 13:30 |
| Much simpler example: https://gist.github.com/parente/a4772297459f05e43e12a5820051431b Wrapping the datetime.datetime.now in a lambda avoids the issue. |
|
|
| msg294482 - (view) |
Author: Louie Lu (louielu) * |
Date: 2017-05-25 14:37 |
| Another way to reproduce this problem: >>> import datetime >>> from collections import defaultdict as dd >>> d = dd(datetime.datetime.now) >>> d['foo'] Traceback (most recent call last): File "", line 1, in SystemError: <built-in method __missing__ of collections.defaultdict object at 0x7f072e55b3b8> returned NULL without setting an error This problem doesn't appear on Python 2.7, too. |
|
|
| msg294484 - (view) |
Author: Xiang Zhang (xiang.zhang) *  |
Date: 2017-05-25 14:53 |
| Interestingly I could only reproduce the failure on 3.6 but not master. |
|
|
| msg294497 - (view) |
Author: Louie Lu (louielu) * |
Date: 2017-05-25 16:59 |
| This bug is introduce at commit: 37e4ef7b17ce6e98ca725c0a53ae14c313c0e48c, then fixed at commit: 998c20962ca3e2e693c1635efe76c0144dde76fc |
|
|
| msg294504 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2017-05-25 17:57 |
| In 3.6 _PyStack_UnpackDict() returns just args if there are no keyword arguments. But if args is NULL, the result of _PyStack_UnpackDict() is treated as error. Thank you for the reproducer Louie! |
|
|
| msg296485 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-06-20 20:34 |
| This bug is a duplicate of bpo-30524. Louie Lu: "This bug is introduce at commit: 37e4ef7b17ce6e98ca725c0a53ae14c313c0e48c, then fixed at commit: 998c20962ca3e2e693c1635efe76c0144dde76fc" This is the fix for master. For Python 3.6, the fix is the commit f0ff849adc6b4a01f9d1f08d9ad0f1511ff84541. I added wrote unit tests for this bug, but also for many ways to calls functions: commit b7577456c430283f8b7ec4e914b701cb943cc69b. > Much simpler example: https://gist.github.com/parente/a4772297459f05e43e12a5820051431b I'm able to reproduce the bug in Python 3.6 before f0ff849adc6b4a01f9d1f08d9ad0f1511ff84541, and also confirm that this commit fixes the bug. The good news is that Python 3.6.2rc1 is already available and contains my fix, and 3.6.2 final should be released soon. |
|
|
| msg296487 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-06-20 20:35 |
| Oh, and thank you Peter Parente for the bug report, and for the script to reproduce the bug. It was usual to validate that the bug is now fixed. |
|
|
| msg297466 - (view) |
Author: Peter Parente (Peter Parente) |
Date: 2017-07-01 01:59 |
| Glad to hear it. Cheers, Victor! |
|
|
| msg297498 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-07-01 22:25 |
| > It was usual to validate that the bug is now fixed. Crap, I wanted to write: useful, not usual!? |
|
|