Issue 30393: test_readline hangs - Python tracker (original) (raw)

Perhaps if you ran the tests in verbose mode, that could narrow down which test it hangs at. Also, if you can capture a KeyboardInterrupt stack trace, that may help. Not sure if it works with your build setup, but maybe you can run something like

./python -m unittest -v test.test_readline

Not too long ago, I added some tests that run a child process in a pseudoterminal (see the “run_pty” function). There were a few platform specific problems getting that working, so there is a good chance your problem may be related.

I wonder if test hangs when you don’t use “--enable-optimizations”?

Output of ./python3 -m unittest -v test.test_readline:

testHistoryUpdates (test.test_readline.TestHistoryManipulation) ... ok test_nonascii_history (test.test_readline.TestHistoryManipulation) ... ok test_write_read_append (test.test_readline.TestHistoryManipulation) ... ok test_auto_history_disabled (test.test_readline.TestReadline) ...

I then found by commenting-out tests that it hangs if and only if run_pty is called, so it seems that your hunch is probably right.

I compiled without --enable-optimizations and the same tests hang. Compiling with that option runs the whole test suite, which revealed the problem, but it doesn't seem to be related.

Here's the KeyboardInterrupt trace:

estHistoryUpdates (test.test_readline.TestHistoryManipulation) ... ok test_nonascii_history (test.test_readline.TestHistoryManipulation) ... ok test_write_read_append (test.test_readline.TestHistoryManipulation) ... ok test_auto_history_disabled (test.test_readline.TestReadline) ... ^CTraceback (most recent call last): File "/home/will/python3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/will/python3.6.1/lib/python3.6/runpy.py", line 85, in run_code exec(code, run_globals) File "/home/will/python3.6.1/lib/python3.6/unittest/main.py", line 18, in main(module=None) File "/home/will/python3.6.1/lib/python3.6/unittest/main.py", line 94, in init self.runTests() File "/home/will/python3.6.1/lib/python3.6/unittest/main.py", line 255, in runTests self.result = testRunner.run(self.test) File "/home/will/python3.6.1/lib/python3.6/unittest/runner.py", line 176, in run test(result) File "/home/will/python3.6.1/lib/python3.6/unittest/suite.py", line 84, in call return self.run(*args, **kwds) File "/home/will/python3.6.1/lib/python3.6/unittest/suite.py", line 122, in run test(result) File "/home/will/python3.6.1/lib/python3.6/unittest/suite.py", line 84, in call return self.run(*args, **kwds) File "/home/will/python3.6.1/lib/python3.6/unittest/suite.py", line 122, in run test(result) File "/home/will/python3.6.1/lib/python3.6/unittest/suite.py", line 84, in call return self.run(*args, **kwds) File "/home/will/python3.6.1/lib/python3.6/unittest/suite.py", line 122, in run test(result) File "/home/will/python3.6.1/lib/python3.6/unittest/case.py", line 649, in call return self.run(*args, **kwds) File "/home/will/python3.6.1/lib/python3.6/unittest/case.py", line 601, in run testMethod() File "/home/will/python3.6.1/lib/python3.6/test/test_readline.py", line 136, in test_auto_history_disabled output = run_pty(self.auto_history_script.format(False)) File "/home/will/python3.6.1/lib/python3.6/test/test_readline.py", line 240, in run_pty for [, events] in sel.select(): File "/home/will/python3.6.1/lib/python3.6/selectors.py", line 323, in select r, w, _ = self._select(self._readers, self._writers, [], timeout) KeyboardInterrupt

Thanks for the information. That narrows the problem down, but I still don’t exactly know why it hangs.

A good workaround may be to put a timeout in the “select” call for a second or so, and if it times out, raise an exception which will fail the test but let the rest of the tests continue. To add the timeout, I would change the code in “run_pty” to look like

selected = sel.select(timeout=1) if not selected: raise Exception("Child timed out: remaining input {!r}, output {!r}".format( input, output)) for [_, events] in selected: ...

This is what “test_auto_history_enabled” is supposed to do:

  1. Create a pseudo-terminal with master and slave file descriptors
  2. Spawn a child process to read from the slave and then print the test result to the slave
  3. Parent waits to be able to read from or write to the PTY master
  4. Parent writes the bytes b"dummy input\r" to the PTY
    1. Child calls “input” which calls the Readline library and reads the input
    2. Child writes the test result to the PTY slave
  5. Parent reads the output from the child
    1. Child closes its slave file descriptor and exits
  6. On Linux, reading then raises EIO indicating all copies of the slave are closed and there is no more output left to read

The parent is hanging at step 3, before entering step 4, 7, or 9. Some input may already have been written to the child, and some output may have been captured, but it assumes the child is still running and is waiting for it to exit, which should close the slave file descriptor.

It would be interesting to confirm if the child has exited or is still waiting for the CR to terminate its input line. Or perhaps the slave file descriptor has somehow been duplicated in a third process without being closed.