| msg145639 - (view) |
Author: Antoine Pitrou (pitrou) *  |
Date: 2011-10-16 20:25 |
| Not sure this is by design or not, but I wanted to report this issue (it's rather hard to diagnose). Here the file descriptor limit is reached, and "import" raises an ImportError while it would be more helpful to let the original OS error slip through (which, here, is EMFILE): >>> import resource >>> resource.setrlimit(resource.RLIMIT_NOFILE, (2, 100)) >>> import encodings.idna Traceback (most recent call last): File "", line 1, in ImportError: No module named 'idna' >>> resource.setrlimit(resource.RLIMIT_NOFILE, (100, 100)) >>> import encodings.idna >>> |
|
|
| msg145640 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-10-16 21:09 |
| _Py_fopen() returns NULL without raising an exception if fopen() fails. Attached patch raises a OSError if fopen() if errno is different than ENOENT. I don't know if ENOENT is the right error number on Windows. Should we ignore more error numbers, like EACCESS? |
|
|
| msg145642 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-10-16 21:19 |
| > _Py_fopen() returns NULL without raising an exception if fopen() fails. > > Attached patch raises a OSError if fopen() if errno is different than > ENOENT. I don't know if ENOENT is the right error number on Windows. It seems so. > Should we ignore more error numbers, like EACCESS? I suggest also ignoring ENOTDIR and EISDIR. Not sure about EACCES. |
|
|
| msg145645 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-10-16 21:36 |
| > I suggest also ignoring ENOTDIR and EISDIR. Not sure about EACCES. We should maybe mimick the io module: >>> open('Lib') IsADirectoryError: [Errno 21] Is a directory: 'Lib' In the io module, opening a directory raises an exception. In the import machinery, an explicit check is done (stat+ISDIR) before opening the file. The file is ignored if it is a directory. We can maybe remove the test before opening the file, and handle IsADirectoryError exception instead. -- I don't think that fopen() can fails with ENOTDIR. rmdir() can fail with ENOTDIR. -- For EACCES: Python ignores currently these errors. I was already surprised by this behaviour. My use case: I have two users (haypo and fusil). I "installed" a module of haypo user into /usr/lib/python... using symlink, but fusil doesn't have access to /home/haypo. When fusil tries to load a module, import fails without any warning or error. Fail with an error is maybe not a good idea because it's not because a directory of sys.path cannot be read, than the import will fail. The module can be found in another (accessible) directory. |
|
|
| msg145646 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-10-16 21:37 |
| > I don't think that fopen() can fails with ENOTDIR. According to POSIX: [ENOTDIR] A component of the path prefix is not a directory, or the filename argument contains at least one non- character and ends with one or more trailing characters and the last pathname component names an existing file that is neither a directory nor a symbolic link to a directory. http://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html |
|
|
| msg145729 - (view) |
Author: Eric Snow (eric.snow) * |
Date: 2011-10-17 17:02 |
| Are you suggesting raising the OSError (or something else) rather than an ImportError? If so, would it make sense to chain the exception instead. That way the existing code that expects ImportError continues to work, while still allowing access to the original error through __cause__. |
|
|
| msg145732 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2011-10-17 17:17 |
| > Are you suggesting raising the OSError (or something else) rather than > an ImportError? If so, would it make sense to chain the exception > instead. That way the existing code that expects ImportError > continues to work Does any existing code rely on ImportError being raised when there's a rare OS error of some sort? |
|
|
| msg181116 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2013-02-01 21:53 |
| Is this still an issue in (at least) 3.4 with the new I/O exceptions? And I feel like there was another bug about this. |
|
|
| msg181135 - (view) |
Author: Eric Snow (eric.snow) * |
Date: 2013-02-02 06:36 |
| Are you thinking of issue 15833? |
|
|
| msg181138 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-02-02 07:21 |
| In 3.3 and 3.4, exceptions are not silenced anymore: >>> import resource >>> resource.setrlimit(resource.RLIMIT_NOFILE, (2, 100)) >>> import encodings.idna Traceback (most recent call last): File "", line 1, in File "<frozen importlib._bootstrap>", line 1584, in _find_and_load File "<frozen importlib._bootstrap>", line 1551, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper File "<frozen importlib._bootstrap>", line 1049, in load_module File "<frozen importlib._bootstrap>", line 1030, in load_module File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper File "<frozen importlib._bootstrap>", line 883, in _load_module File "<frozen importlib._bootstrap>", line 1008, in get_code File "<frozen importlib._bootstrap>", line 1058, in get_data OSError: [Errno 24] Too many open files: '/home/antoine/cpython/default/Lib/encodings/idna.py' Given that fixing the issue in bugfix branches would be a slight behaviour change, I think we can close this issue. |
|
|