Issue 21951: tcl test change crashes AIX (original) (raw)

Created on 2014-07-10 17:38 by David.Edelsohn, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (26)

msg222680 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-07-10 17:38

The patch for Issue #21881 causes CPython test_tcl to crash on AIX.

$ ./python -m test -v test_tcl == CPython 3.5.0a0 (default:d1f89eb9ea1e+, Jul 10 2014, 10:21:22) [GCC 4.8.1] == AIX-1-00F84C0C4C00-powerpc-32bit big-endian == hash algorithm: siphash24 32bit == /home/dje/src/cpython/build/test_python_27984522 Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0) [1/1] test_tcl patchlevel = 8.4.7 testCall (test.test_tcl.TclTest) ... ok testCallException (test.test_tcl.TclTest) ... ok testCallException2 (test.test_tcl.TclTest) ... ok testEval (test.test_tcl.TclTest) ... ok testEvalException (test.test_tcl.TclTest) ... ok testEvalException2 (test.test_tcl.TclTest) ... ok testEvalFile (test.test_tcl.TclTest) ... ok testEvalFileException (test.test_tcl.TclTest) ... ok testGetVar (test.test_tcl.TclTest) ... ok testGetVarArray (test.test_tcl.TclTest) ... ok testGetVarArrayException (test.test_tcl.TclTest) ... ok testGetVarException (test.test_tcl.TclTest) ... ok testLoadWithUNC (test.test_tcl.TclTest) ... skipped 'Requires Windows' testPackageRequireException (test.test_tcl.TclTest) ... ok testSetVar (test.test_tcl.TclTest) ... ok testSetVarArray (test.test_tcl.TclTest) ... ok testUnsetVar (test.test_tcl.TclTest) ... ok testUnsetVarArray (test.test_tcl.TclTest) ... ok testUnsetVarException (test.test_tcl.TclTest) ... ok test_eval_null_in_result (test.test_tcl.TclTest) ... ok test_evalfile_null_in_result (test.test_tcl.TclTest) ... ok test_exprboolean (test.test_tcl.TclTest) ... ok test_exprdouble (test.test_tcl.TclTest) ... ok test_exprlong (test.test_tcl.TclTest) ... ok test_exprstring (test.test_tcl.TclTest) ... ok test_getboolean (test.test_tcl.TclTest) ... ok test_getdouble (test.test_tcl.TclTest) ... ok test_getint (test.test_tcl.TclTest) ... ok test_passing_values (test.test_tcl.TclTest) ... ok test_split (test.test_tcl.TclTest) ... ok test_splitlist (test.test_tcl.TclTest) ... ok test_user_command (test.test_tcl.TclTest) ... Assertion failed: __EX, file Objects/methodobject.c, line 94 Fatal Python error: Aborted

Current thread 0x00000001 (most recent call first): File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 426 in check File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 458 in test_user_command File "/home/dje/src/cpython/Lib/unittest/case.py", line 577 in run File "/home/dje/src/cpython/Lib/unittest/case.py", line 625 in call File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in call File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in call File "/home/dje/src/cpython/Lib/unittest/runner.py", line 168 in run File "/home/dje/src/cpython/Lib/test/support/init.py", line 1724 in _run_suite File "/home/dje/src/cpython/Lib/test/support/init.py", line 1758 in run_unittest File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 601 in test_main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1278 in runtest_inner File "/home/dje/src/cpython/Lib/test/regrtest.py", line 978 in runtest File "/home/dje/src/cpython/Lib/test/regrtest.py", line 763 in main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1562 in main_in_temp_cwd File "/home/dje/src/cpython/Lib/test/main.py", line 3 in File "/home/dje/src/cpython/Lib/runpy.py", line 85 in _run_code File "/home/dje/src/cpython/Lib/runpy.py", line 170 in _run_module_as_main IOT/Abort trap (core dumped)

msg222837 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-07-12 12:34

Is it crashes when comment out "check((), '')"? Is it crashes when comment out two previous checks (for inf and -inf)?

msg222897 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-07-13 00:02

If I comment out all three tests, it runs.

--- a/Lib/test/test_tcl.py Thu Jul 10 01:17:11 2014 -0400 +++ b/Lib/test/test_tcl.py Sat Jul 12 16:59:33 2014 -0700 @@ -455,9 +455,9 @@ check(float('inf'), 'Inf', eq=float_eq) check(-float('inf'), '-Inf', eq=float_eq) # XXX NaN representation can be not parsable by float()

```
   check((), '')
```

   check((1, (2,), (3, 4), '5 6', ()), '1 2 {3 4} {5 6} {}')

   check([1, [2,], [3, 4], '5 6', []], '1 2 {3 4} {5 6} {}')

```
   #check((), '')
```

   #check((1, (2,), (3, 4), '5 6', ()), '1 2 {3 4} {5 6} {}')

```
   #check([1, [2,], [3, 4], '5 6', []], '1 2 {3 4} {5 6} {}')
```
def test_splitlist(self): splitlist = self.interp.tk.splitlist

msg222950 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-07-13 16:23

Is following test passed?

    check((1,), '1')

Is CPython crashes when change signature of testfunc?

--- a/Lib/test/test_tcl.py Sat Jul 12 18:26:03 2014 +0300 +++ b/Lib/test/test_tcl.py Sun Jul 13 19:21:55 2014 +0300 @@ -416,7 +416,7 @@

 def test_user_command(self):
     result = None

```
   def testfunc(arg):
```

   def testfunc(arg=None, *args):
       nonlocal result
       result = arg
       return arg

msg222965 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-07-13 18:53

Is following test passed?

    check((1,), '1')

That test succeeds.

Is CPython crashes when change signature of testfunc?

--- a/Lib/test/test_tcl.py Sat Jul 12 18:26:03 2014 +0300 +++ b/Lib/test/test_tcl.py Sun Jul 13 19:21:55 2014 +0300 @@ -416,7 +416,7 @@

 def test_user_command(self):
     result = None

```
   def testfunc(arg):
```

   def testfunc(arg=None, *args):
       nonlocal result
       result = arg
       return arg

This change does not have any effect on the original failure. The assertion failure and crash continues to occur.

msg222968 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-07-13 19:26

#5 0x1019bb90 in PyCFunction_Call (func=0x30533b5c, arg=0x305d8ab4, kw=0x0) at Objects/methodobject.c:94 #6 0x1012534c in call_function (pp_stack=0x2ff16144, oparg=2) at Python/ceval.c:4269

94 CHECK_RESULT(res);

(gdb) print res $9 = (PyObject *) 0x0 (gdb) print meth $10 = (PyCFunction) @0x20380888: 0xd8dd57dc (gdb) print self $11 = (PyObject *) 0x305d44d8

msg222970 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-07-13 19:53

May be this patch will help.

msg222977 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-07-13 21:49

No difference with the patch. The problem is not a a malloc() failure.

msg224290 - (view)

Author: Roundup Robot (python-dev) (Python triager)

Date: 2014-07-30 08:02

New changeset 7de64def6565 by Serhiy Storchaka in branch '2.7': Issue #21951: Temporary skip crashing test_user_command on AIX. http://hg.python.org/cpython/rev/7de64def6565

New changeset 31f4cb1fede9 by Serhiy Storchaka in branch '3.4': Issue #21951: Temporary skip crashing test_user_command on AIX. http://hg.python.org/cpython/rev/31f4cb1fede9

New changeset de32cd419174 by Serhiy Storchaka in branch 'default': Issue #21951: Temporary skip crashing test_user_command on AIX. http://hg.python.org/cpython/rev/de32cd419174

msg224291 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-07-30 08:07

Link to log with a crash:

http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2442/steps/test/logs/stdio

msg226345 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-04 07:59

The failing assertion means that the method raised an exception and returned a result, or failed but didn't raised an exception. According to the gdb trace, the function failed (res is NULL). You can display the current exception in gdb using: "print PyErr_Occurred()" (NULL if there is no exception).

Please run test_tcl on AIX with attached test_tkinter.patch applied.

msg226359 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-09-04 12:00

As far as I remember, both res and PyErr_Occurred() were NULL.

msg226361 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-09-04 14:22

When the failing test is re-enabled, the _tkinter patch produces the following output:

test_user_command (test.test_tcl.TclTest) ... Assertion failed: __EX, file /home/dje/src/cpython/Modules/_tkinter.c, line 1277 Fatal Python error: Aborted

Current thread 0x00000001 (most recent call first): File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 435 in check File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 468 in test_user_command File "/home/dje/src/cpython/Lib/unittest/case.py", line 577 in run File "/home/dje/src/cpython/Lib/unittest/case.py", line 625 in call File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in call File "/home/dje/src/cpython/Lib/unittest/suite.py", line 125 in run File "/home/dje/src/cpython/Lib/unittest/suite.py", line 87 in call File "/home/dje/src/cpython/Lib/unittest/runner.py", line 168 in run File "/home/dje/src/cpython/Lib/test/support/init.py", line 1750 in _run_suite File "/home/dje/src/cpython/Lib/test/support/init.py", line 1784 in run_unittest File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 615 in test_main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1280 in runtest_inner File "/home/dje/src/cpython/Lib/test/regrtest.py", line 978 in runtest File "/home/dje/src/cpython/Lib/test/regrtest.py", line 763 in main File "/home/dje/src/cpython/Lib/test/regrtest.py", line 1564 in main_in_temp_cwd File "/home/dje/src/cpython/Lib/test/main.py", line 3 in File "/home/dje/src/cpython/Lib/runpy.py", line 85 in _run_code File "/home/dje/src/cpython/Lib/runpy.py", line 170 in _run_module_as_main IOT/Abort trap (core dumped)

msg226362 - (view)

Author: Roundup Robot (python-dev) (Python triager)

Date: 2014-09-04 15:30

New changeset 9ab404cdcaa1 by Victor Stinner in branch 'default': Issue #21951: Fix AsObj() of the _tkinter module: raise MemoryError on memory http://hg.python.org/cpython/rev/9ab404cdcaa1

msg226364 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-04 15:47

When the failing test is re-enabled, the _tkinter patch produces the following output:

Thanks. I found and fixed a bug in AsObj() for tuple and list. It looks like ckalloc(size) returns NULL if size is 0. Please update Python, compile and run test_tcl to see if you get a MemoryError exception instead of a crash.

I just created the issue #22336 which may fix this issue if my analysis was correct. It uses PyMem_Malloc(size) which has a well-defined behaviour when size is 0.

msg226367 - (view)

Author: David Edelsohn (David.Edelsohn) *

Date: 2014-09-04 17:19

$ ./python -m test -v test_tcl == CPython 3.5.0a0 (default:9ab404cdcaa1+, Sep 4 2014, 10:06:33) [GCC 4.8.1] == AIX-1-00F84C0C4C00-powerpc-32bit big-endian == hash algorithm: siphash24 32bit == /home/dje/src/cpython/build/test_python_48694044 Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0) [1/1] test_tcl patchlevel = 8.4.7 testCall (test.test_tcl.TclTest) ... ok testCallException (test.test_tcl.TclTest) ... ok testCallException2 (test.test_tcl.TclTest) ... ok testEval (test.test_tcl.TclTest) ... ok testEvalException (test.test_tcl.TclTest) ... ok testEvalException2 (test.test_tcl.TclTest) ... ok testEvalFile (test.test_tcl.TclTest) ... ok testEvalFileException (test.test_tcl.TclTest) ... ok testGetVar (test.test_tcl.TclTest) ... ok testGetVarArray (test.test_tcl.TclTest) ... ok testGetVarArrayException (test.test_tcl.TclTest) ... ok testGetVarException (test.test_tcl.TclTest) ... ok testLoadWithUNC (test.test_tcl.TclTest) ... skipped 'Requires Windows' testPackageRequireException (test.test_tcl.TclTest) ... ok testSetVar (test.test_tcl.TclTest) ... ok testSetVarArray (test.test_tcl.TclTest) ... ok testUnsetVar (test.test_tcl.TclTest) ... ok testUnsetVarArray (test.test_tcl.TclTest) ... ok testUnsetVarException (test.test_tcl.TclTest) ... ok test_eval_null_in_result (test.test_tcl.TclTest) ... ok test_evalfile_null_in_result (test.test_tcl.TclTest) ... ok test_exprboolean (test.test_tcl.TclTest) ... ok test_exprdouble (test.test_tcl.TclTest) ... ok test_exprlong (test.test_tcl.TclTest) ... ok test_exprstring (test.test_tcl.TclTest) ... ok test_getboolean (test.test_tcl.TclTest) ... ok test_getdouble (test.test_tcl.TclTest) ... ok test_getint (test.test_tcl.TclTest) ... ok test_passing_values (test.test_tcl.TclTest) ... ok test_split (test.test_tcl.TclTest) ... ok test_splitlist (test.test_tcl.TclTest) ... ok test_user_command (test.test_tcl.TclTest) ... ERROR testFlattenLen (test.test_tcl.TkinterTest) ... ok test_huge_string_builtins (test.test_tcl.BigmemTclTest) ... skipped 'needs UINT_MAX < SIZE_MAX' test_huge_string_call (test.test_tcl.BigmemTclTest) ... skipped 'needs UINT_MAX < SIZE_MAX'

====================================================================== ERROR: test_user_command (test.test_tcl.TclTest)

Traceback (most recent call last): File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 468, in test_user_command check((), '') File "/home/dje/src/cpython/Lib/test/test_tcl.py", line 435, in check r = self.interp.call('testfunc', value) MemoryError

Ran 35 tests in 0.133s

FAILED (errors=1, skipped=3) test test_tcl failed 1 test failed: test_tcl

msg226374 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-04 19:41

Thanks David! Your ouput confirms my analysis. asobj_empty_sequence.patch should fix the test on AIX.

I read Tcl source code, Tcl_NewListObj() doesn't use the second parameter if the first parameter (length) is zero.

msg226400 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-09-05 06:15

The patch can be simpler. And there is yet one similar bug, for unicode strings.

msg226637 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-09-09 14:59

As Victor noted on IRC, ckalloc() panics and doesn't returns NULL in case of error. We should use attemptckalloc() instead.

msg226721 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-10 21:24

I'm not sure that tkinter_ckallock.patch is correct. Extract of attemptckalloc() manual page: "If the allocation fails, these functions will return NULL. Note that on some platforms, but not all, attempting to allocate a zero-sized block of memory will also cause these functions to return NULL."

http://linux.die.net/man/3/attemptckalloc

It looks like you are fixing two different issues in the same patch: fix AIX and enhance handling of memory allocation failure.

Can you please split your patch in two parts? For AIX, adding these two lines for list/tuple should be enough:

if (size == 0) return Tcl_NewListObj(0, NULL);

msg226723 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-10 21:26

"I'm not sure that tkinter_ckallock.patch is correct."

Oh, when I read my message, I realized that I was not explicit enough. It looks like attemptckalloc() can be called with 0 at some places with tkinter_ckallock.patch. But I didn't check carefully, the patch is maybe correct.

msg226751 - (view)

Author: Roundup Robot (python-dev) (Python triager)

Date: 2014-09-11 08:04

New changeset ee969a717cb5 by Serhiy Storchaka in branch '2.7': Issue #21951: Use attemptckalloc() instead of ckalloc() in Tkinter. http://hg.python.org/cpython/rev/ee969a717cb5

New changeset 1223c882253f by Serhiy Storchaka in branch '3.4': Issue #21951: Use attemptckalloc() instead of ckalloc() in Tkinter. http://hg.python.org/cpython/rev/1223c882253f

New changeset 499b60b7d067 by Serhiy Storchaka in branch 'default': Issue #21951: Use attemptckalloc() instead of ckalloc() in Tkinter. http://hg.python.org/cpython/rev/499b60b7d067

New changeset d6c7ab5a2065 by Serhiy Storchaka in branch '2.7': Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with http://hg.python.org/cpython/rev/d6c7ab5a2065

New changeset 6a96c28f9474 by Serhiy Storchaka in branch '3.4': Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with http://hg.python.org/cpython/rev/6a96c28f9474

New changeset 7b7bae546959 by Serhiy Storchaka in branch 'default': Issue #21951: Fixed a crash in Tkinter on AIX when called Tcl command with http://hg.python.org/cpython/rev/7b7bae546959

msg226754 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-09-11 08:24

Committed these two changes as separate patches.

msg226762 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-11 11:05

Committed these two changes as separate patches.

Thanks, it's clearer like that.

msg226763 - (view)

Author: STINNER Victor (vstinner) * (Python committer)

Date: 2014-09-11 11:07

test_tcl now pass on AIX: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2592/steps/test/logs/stdio

Thanks Serhiy for the fix.

msg226765 - (view)

Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer)

Date: 2014-09-11 11:35

Thank you Victor for great investigation of this issue.

History

Date

User

Action

Args

2022-04-11 14:58:05

admin

set

github: 66150

2014-09-11 11:35:34

serhiy.storchaka

set

messages: +

2014-09-11 11:07:28

vstinner

set

status: open -> closed
resolution: fixed
messages: +

2014-09-11 11:05:11

vstinner

set

messages: +

2014-09-11 08:24:27

serhiy.storchaka

set

messages: +
stage: patch review -> commit review

2014-09-11 08:04:30

python-dev

set

messages: +

2014-09-10 21:26:55

vstinner

set

messages: +

2014-09-10 21:24:04

vstinner

set

messages: +

2014-09-09 14:59:56

serhiy.storchaka

set

files: + tkinter_ckallock.patch

messages: +
stage: needs patch -> patch review

2014-09-09 14:51:07

serhiy.storchaka

set

assignee: serhiy.storchaka

2014-09-05 06:15:51

serhiy.storchaka

set

files: + asobj_empty_sequence_2.patch

messages: +

2014-09-04 19:41:28

vstinner

set

files: + asobj_empty_sequence.patch

messages: +

2014-09-04 17:19:41

David.Edelsohn

set

messages: +

2014-09-04 15:47:22

vstinner

set

messages: +

2014-09-04 15:30:33

python-dev

set

messages: +

2014-09-04 14:22:11

David.Edelsohn

set

messages: +

2014-09-04 12:00:45

serhiy.storchaka

set

messages: +

2014-09-04 07:59:05

vstinner

set

files: + tkinter_debug.patch

messages: +

2014-07-30 08:07:23

serhiy.storchaka

set

messages: +
stage: patch review -> needs patch

2014-07-30 08:02:14

python-dev

set

nosy: + python-dev
messages: +

2014-07-13 21:49:04

David.Edelsohn

set

messages: +

2014-07-13 19:53:51

serhiy.storchaka

set

files: + tkinter_nomemory.patch
keywords: + patch
messages: +

stage: patch review

2014-07-13 19:26:51

David.Edelsohn

set

messages: +

2014-07-13 18:53:03

David.Edelsohn

set

messages: +

2014-07-13 16:23:27

serhiy.storchaka

set

messages: +

2014-07-13 00:02:05

David.Edelsohn

set

messages: +

2014-07-12 12:34:14

serhiy.storchaka

set

messages: +

2014-07-12 12:21:43

serhiy.storchaka

set

priority: normal -> high

2014-07-10 17:38:46

David.Edelsohn

create