| msg313487 - (view) |
Author: Nathan Henrie (n8henrie) * |
Date: 2018-03-09 16:53 |
| Failing for me on latest 3.6, 3.6.1, 3.5.5, may be related to https://bugs.python.org/issue32517, presumably a change on macOS KQueue stuff. Can anyone else on macOS 10.13.3 see if they can reproduce? ``` make clean && ./configure --with-pydebug && make -j ./python.exe -m unittest -v test.test_selectors.PollSelectorTestCase ``` |
|
|
| msg313488 - (view) |
Author: Nathan Henrie (n8henrie) * |
Date: 2018-03-09 16:55 |
| Traceback: ``` File "cpython/Lib/test/test_selectors.py", line 453, in test_above_fd_setsize self.assertEqual(NUM_FDS // 2, len(s.select())) File "cpython/Lib/selectors.py", line 376, in select fd_event_list = self._poll.poll(timeout) OSError: [Errno 22] Invalid argument ``` |
|
|
| msg313489 - (view) |
Author: Ned Deily (ned.deily) *  |
Date: 2018-03-09 17:26 |
| Sorry, I am not seeing this failure on 10.13.3 with either current 3.6 head or with v3.6.1. Didn't try anything else. |
|
|
| msg313840 - (view) |
Author: Nathan Henrie (n8henrie) * |
Date: 2018-03-14 20:50 |
| Hmmm, still failing for me. I wonder if it's something specific to my machine. ``` git reset --hard 3.6 && make clean && git pull && ./configure --with-pydebug && make -j && ./python.exe -m unittest -v test.test_selectors.PollSelectorTestCase.test_above_fd_setsize ``` Related: - https://bugs.python.org/issue18963 - https://bugs.python.org/issue21901 |
|
|
| msg313844 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2018-03-14 23:20 |
| I'm not sure what to suggest other than taking a close look at and instrumenting that part of test_selectors. One thing you could try is seeing what the call to resource.getrlimit(resource.RLIMIT_NOFILE) is returning. You can also look at the values for your system's kernel by using the sysctl utility; see man sysctl. For example, on my system: >>> resource.getrlimit(resource.RLIMIT_NOFILE) (7168, 9223372036854775807) $ sysctl kern.maxfilesperproc kern.maxfilesperproc: 36864 That may or may not be relevant. Long shot: are you running under an account that does not have administrator privs? |
|
|
| msg313856 - (view) |
Author: Nathan Henrie (n8henrie) * |
Date: 2018-03-15 08:09 |
| Thanks for the response -- I'll keep looking, feel free to close since it's not being reproduced. ``` $ sysctl kern.maxfilesperproc kern.maxfilesperproc: 64000 $ ./python.exe -c 'import resource; print(resource.getrlimit(resource.RLIMIT_NOFILE))' (64000, 524288) ``` Nope, admin access. |
|
|
| msg313951 - (view) |
Author: Nathan Henrie (n8henrie) * |
Date: 2018-03-16 15:08 |
| Update -- I found the following plist at `/Library/LaunchDaemons/limit.maxfiles.plist`: ```xml Label limit.maxfiles ProgramArguments launchctl limit maxfiles 64000 524288 RunAtLoad ServiceIPC ``` I think I made this file at some point to deal with an error about insufficient file descriptors while playing with asyncio / sockets. After `launchctl unload`ing it and rebooting **the test now passes**, and I now see a *much* higher hard limit (similar to that posted by Ned) with the `getrlimit()` command, and `unlimited` as the hard limit with `launchtl limit`. I can reproduce the failure by restoring my prior settings: ```bash $ sudo launchctl limit maxfiles 64000 524288 $ ulimit -n 64000 $ $ ./python.exe -m unittest -v test.test_selectors.PollSelectorTestCase.test_above_fd_setsize test_above_fd_setsize (test.test_selectors.PollSelectorTestCase) ... ERROR ====================================================================== ERROR: test_above_fd_setsize (test.test_selectors.PollSelectorTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/n8henrie/git/cpython/Lib/test/support/__init__.py", line 600, in wrapper return func(*args, **kw) File "/Users/n8henrie/git/cpython/Lib/test/test_selectors.py", line 453, in test_above_fd_setsize self.assertEqual(NUM_FDS // 2, len(s.select())) File "/Users/n8henrie/git/cpython/Lib/selectors.py", line 376, in select fd_event_list = self._poll.poll(timeout) OSError: [Errno 22] Invalid argument ---------------------------------------------------------------------- Ran 1 test in 9.771s FAILED (errors=1) ``` |
|
|
| msg313952 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2018-03-16 15:49 |
| Glad you found the problem - and that it wasn't a Python bug! |
|
|
| msg314149 - (view) |
Author: Nathan Henrie (n8henrie) * |
Date: 2018-03-20 16:06 |
| I've continued looking into this. If you have your limits configured higher than default, as I did (and which seems to be working fine): ``` sudo launchctl limit maxfiles 64000 524288 ulimit -Sn 64000 ``` then you'll find that having a soft limit (`NUM_FDS`, prior to the `-=32`) greater than 10273 causes the error I was seeing. First, I found this number by passing in NUM_FDS as an envvar to the unittest and looping in a bash script, but I have since extracted some of the relevant portions to a separate python script: ```python import selectors import socket import sys def main(NUM_FDS): s = selectors.PollSelector() NUM_FDS -= 32 sockets = [] try: for i in range(NUM_FDS // 2): try: rd, wr = socket.socketpair() sockets.extend([rd, wr]) except OSError as e: print(f"err 2: {type(e)} {e}") # too many FDs, skip - note that we should only catch EMFILE # here, but apparently *BSD and Solaris can fail upon connect() # or bind() with EADDRNOTAVAIL, so let's be safe print("FD limit reached") try: s.register(rd, selectors.EVENT_READ) s.register(wr, selectors.EVENT_WRITE) except OSError as e: print(f"err 3: {type(e)} {e}") if e.errno == errno.ENOSPC: # this can be raised by epoll if we go over # fs.epoll.max_user_watches sysctl print("FD limit reached") raise try: slen = len(s.select()) except OSError as e: print(f"err 4: {type(e)} {e}") raise print(f"asserting {NUM_FDS // 2} == {slen}") assert NUM_FDS // 2 == slen finally: for sock in sockets: sock.close() s.close() print("Closed") if __name__ == "__main__": main(int(sys.argv[1])) ``` ```shellsession $ ./python.exe tester.py 10273 asserting 5120 == 5120 Closed $ ./python.exe tester.py 10274 err 4: <class 'OSError'> [Errno 22] Invalid argument Closed Traceback (most recent call last): File "tester.py", line 50, in main(int(sys.argv[1])) File "tester.py", line 36, in main slen = len(s.select()) File "cpython/Lib/selectors.py", line 376, in select fd_event_list = self._poll.poll(timeout) OSError: [Errno 22] Invalid argument ``` Tested and confirmed that I can provoke the error with the above launchctl / ulimit settings on my wife's Macbook Air. I wonder if the 10273 limit I'm running up against has something to do with the 10240 number I keep running into as a default kern.maxfilesperproc (https://unix.stackexchange.com/a/350824/77904), given the `NUM_FDS -= 32` in the test. |
|
|