Issue 15765: test_getcwd_long_pathnames (in test_posix) kills NetBSD (original) (raw)

Created on 2012-08-22 14:12 by trent, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (7)

msg168885 - (view)

Author: Trent Nelson (trent) *

Date: 2012-08-22 14:12

The 2.7 builds of both my NetBSD slaves keep hanging on test_posix (and eventually time out after an hour). Running the test manually:

netbsd51-x86-1$ pwd
/home/cpython/2.7 netbsd51-x86-1$ b/test/regrtest.py -uall -rwW -v test_posix < == CPython 2.7.3+ (2.7:c1c45755397b, Aug 22 2012, 14:05:02) [GCC 4.1.3 20080704 prerelease (NetBSD nb2 20081120)] == NetBSD-5.1.2-i386-32bit-ELF little-endian == /home/cpython/2.7/build/test_python_10630 Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=1) Using random seed 2040018 test_posix testNoArgFunctions (test.test_posix.PosixTester) ... ok test_access (test.test_posix.PosixTester) ... ok test_chdir (test.test_posix.PosixTester) ... ok test_chflags (test.test_posix.PosixTester) ... ok test_chown (test.test_posix.PosixTester) ... ok test_confstr (test.test_posix.PosixTester) ... ok test_dup (test.test_posix.PosixTester) ... ok test_dup2 (test.test_posix.PosixTester) ... ok test_fchown (test.test_posix.PosixTester) ... ok test_fdopen (test.test_posix.PosixTester) ... ok test_fstat (test.test_posix.PosixTester) ... ok test_fstatvfs (test.test_posix.PosixTester) ... ok test_ftruncate (test.test_posix.PosixTester) ... ok test_getcwd_long_pathnames (test.test_posix.PosixTester) ... load: 0.50 cmd: python 10630 [runnable] 0.27u 13.07s 46% 10976k load: 0.87 cmd: python 10630 [runnable] 0.27u 94.24s 98% 10976k ^C^C

Placeholder issue at the moment.

....although this looks pretty dodgy:

                os.chdir(dirname)
                try:
                    os.getcwd()
                    if current_path_length < 4099:
                        _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)

msg168902 - (view)

Author: Stefan Krah (skrah) *

Date: 2012-08-22 17:51

The timeouts on NetBSD and OpenBSD are very likely related to bugs in the user-space thread libraries. See also #8714.

I would recommend running these bots --without-threads.

msg168994 - (view)

Author: Charles-François Natali (neologix) *

Date: 2012-08-24 10:13

This doesn't look thread-related: """ load: 0.50 cmd: python 10630 [runnable] 0.27u 13.07s 46% 10976k load: 0.87 cmd: python 10630 [runnable] 0.27u 94.24s 98% 10976k """

See the huge system time? This probably means that the kernel is struggling with path name resolution, although that's surprising because 4099 is a reasonable limit. Do you have something like strace or truss on NetBSD? The first thing to try could be to lower the limit, and see what happens, but it's really likely a FS scalibility limit you're hitting (altough, once again, that's really surprising given the reasonable limit). Are you running those under VMs?

msg169012 - (view)

Author: Trent Nelson (trent) *

Date: 2012-08-24 12:53

Hacked the test to print out info each loop:

test_getcwd_long_pathnames (test.test_posix.PosixTester) ... [0/0] getcwd-test-directory-0123456789abcdef-01234567890abcdef [1/57] getcwd-test-directory-0123456789abcdef-01234567890abcdef [2/114] getcwd-test-directory-0123456789abcdef-01234567890abcdef [3/171] getcwd-test-directory-0123456789abcdef-01234567890abcdef [4/228] getcwd-test-directory-0123456789abcdef-01234567890abcdef [5/285] getcwd-test-directory-0123456789abcdef-01234567890abcdef [6/342] getcwd-test-directory-0123456789abcdef-01234567890abcdef [7/399] getcwd-test-directory-0123456789abcdef-01234567890abcdef [8/456] getcwd-test-directory-0123456789abcdef-01234567890abcdef [9/513] getcwd-test-directory-0123456789abcdef-01234567890abcdef [10/570] getcwd-test-directory-0123456789abcdef-01234567890abcdef [11/627] getcwd-test-directory-0123456789abcdef-01234567890abcdef [12/684] getcwd-test-directory-0123456789abcdef-01234567890abcdef [13/741] getcwd-test-directory-0123456789abcdef-01234567890abcdef [14/798] getcwd-test-directory-0123456789abcdef-01234567890abcdef [15/855] getcwd-test-directory-0123456789abcdef-01234567890abcdef [16/912] getcwd-test-directory-0123456789abcdef-01234567890abcdef [17/969] getcwd-test-directory-0123456789abcdef-01234567890abcdef [18/1026] getcwd-test-directory-0123456789abcdef-01234567890abcdef [19/1083] getcwd-test-directory-0123456789abcdef-01234567890abcdef [20/1140] getcwd-test-directory-0123456789abcdef-01234567890abcdef [21/1197] getcwd-test-directory-0123456789abcdef-01234567890abcdef [22/1254] getcwd-test-directory-0123456789abcdef-01234567890abcdef [23/1311] getcwd-test-directory-0123456789abcdef-01234567890abcdef [24/1368] getcwd-test-directory-0123456789abcdef-01234567890abcdef [25/1425] getcwd-test-directory-0123456789abcdef-01234567890abcdef [26/1482] getcwd-test-directory-0123456789abcdef-01234567890abcdef [27/1539] getcwd-test-directory-0123456789abcdef-01234567890abcdef [28/1596] getcwd-test-directory-0123456789abcdef-01234567890abcdef [29/1653] getcwd-test-directory-0123456789abcdef-01234567890abcdef [30/1710] getcwd-test-directory-0123456789abcdef-01234567890abcdef [31/1767] getcwd-test-directory-0123456789abcdef-01234567890abcdef [32/1824] getcwd-test-directory-0123456789abcdef-01234567890abcdef [33/1881] getcwd-test-directory-0123456789abcdef-01234567890abcdef [34/1938] getcwd-test-directory-0123456789abcdef-01234567890abcdef [35/1995] getcwd-test-directory-0123456789abcdef-01234567890abcdef [36/2052] getcwd-test-directory-0123456789abcdef-01234567890abcdef [37/2109] getcwd-test-directory-0123456789abcdef-01234567890abcdef [38/2166] getcwd-test-directory-0123456789abcdef-01234567890abcdef [39/2223] getcwd-test-directory-0123456789abcdef-01234567890abcdef [40/2280] getcwd-test-directory-0123456789abcdef-01234567890abcdef [41/2337] getcwd-test-directory-0123456789abcdef-01234567890abcdef [42/2394] getcwd-test-directory-0123456789abcdef-01234567890abcdef [43/2451] getcwd-test-directory-0123456789abcdef-01234567890abcdef [44/2508] getcwd-test-directory-0123456789abcdef-01234567890abcdef [45/2565] getcwd-test-directory-0123456789abcdef-01234567890abcdef [46/2622] getcwd-test-directory-0123456789abcdef-01234567890abcdef [47/2679] getcwd-test-directory-0123456789abcdef-01234567890abcdef [48/2736] getcwd-test-directory-0123456789abcdef-01234567890abcdef [49/2793] getcwd-test-directory-0123456789abcdef-01234567890abcdef [50/2850] getcwd-test-directory-0123456789abcdef-01234567890abcdef [51/2907] getcwd-test-directory-0123456789abcdef-01234567890abcdef [52/2964] getcwd-test-directory-0123456789abcdef-01234567890abcdef [53/3021] getcwd-test-directory-0123456789abcdef-01234567890abcdef [54/3078] getcwd-test-directory-0123456789abcdef-01234567890abcdef [55/3135] getcwd-test-directory-0123456789abcdef-01234567890abcdef [56/3192] getcwd-test-directory-0123456789abcdef-01234567890abcdef [57/3249] getcwd-test-directory-0123456789abcdef-01234567890abcdef [58/3306] getcwd-test-directory-0123456789abcdef-01234567890abcdef [59/3363] getcwd-test-directory-0123456789abcdef-01234567890abcdef [60/3420] getcwd-test-directory-0123456789abcdef-01234567890abcdef [61/3477] getcwd-test-directory-0123456789abcdef-01234567890abcdef [62/3534] getcwd-test-directory-0123456789abcdef-01234567890abcdef [63/3591] getcwd-test-directory-0123456789abcdef-01234567890abcdef [64/3648] getcwd-test-directory-0123456789abcdef-01234567890abcdef [65/3705] getcwd-test-directory-0123456789abcdef-01234567890abcdef [66/3762] getcwd-test-directory-0123456789abcdef-01234567890abcdef [67/3819] getcwd-test-directory-0123456789abcdef-01234567890abcdef [68/3876] getcwd-test-directory-0123456789abcdef-01234567890abcdef [69/3933] getcwd-test-directory-0123456789abcdef-01234567890abcdef [70/3990] getcwd-test-directory-0123456789abcdef-01234567890abcdef ^C^C load: 1.08 cmd: python 4486 [runnable] 0.07u 14.38s 40% 10012k load: 1.08 cmd: python 4486 [runnable] 0.07u 14.91s 40% 10012k load: 1.08 cmd: python 4486 [runnable] 0.07u 15.24s 42% 10012k load: 1.08 cmd: python 4486 [runnable] 0.07u 15.40s 42% 10012k Terminated

(Had to kill it from a different process, mind you; the ctrl-Cs never get caught.)

Yeah, it's a VM. Also, it looks like 3.x uses 1027 instead of 4099.

It's definitely odd that it ends up spinning in the kernel at ~3990. I'll do some more digging.

msg169027 - (view)

Author: Trent Nelson (trent) *

Date: 2012-08-24 15:25

Looks like this is a duplicate of issue 9185. The fix used for that just needs to be extended to cover NetBSD as well. See proposed patch. I'll commit this to 2.7 in a day or two if there are no objections.

Antoine: added you as nosy as you were involved with the original patch.

msg169119 - (view)

Author: Stefan Krah (skrah) *

Date: 2012-08-25 09:21

I completely forgot about that issue. The patch looks good to me.

msg169375 - (view)

Author: Roundup Robot (python-dev)

Date: 2012-08-29 13:21

New changeset bfa5d0ddfbeb by Trent Nelson in branch '2.7': Issue #15765: Fix quirky NetBSD getcwd() behaviour. http://hg.python.org/cpython/rev/bfa5d0ddfbeb

History

Date

User

Action

Args

2022-04-11 14:57:35

admin

set

github: 59969

2012-08-30 05:32:25

trent

set

status: open -> closed

2012-08-29 13:21:34

python-dev

set

nosy: + python-dev
messages: +

2012-08-25 09:21:00

skrah

set

messages: +

2012-08-24 18:07:15

neologix

link

issue11225 superseder

2012-08-24 15:25:44

trent

set

files: + getcwd_netbsd.patch
resolution: duplicate
messages: +

keywords: + patch

2012-08-24 12:53:44

trent

set

messages: +

2012-08-24 10:13:17

neologix

set

messages: +

2012-08-23 16:20:39

vstinner

set

nosy: + vstinner, neologix

2012-08-22 17:51:50

skrah

set

nosy: + skrah
messages: +

2012-08-22 14:12:15

trent

create