Issue 11914: pydoc modules/help('modules') crash in dirs with unreadable subdirs (original) (raw)
Created on 2011-04-24 03:34 by okopnik, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (6)
Author: Ben Okopnik (okopnik)
Date: 2011-04-24 03:34
Long-standing problem (happens in every Python version I've tested). The usual situation is when invoking Python (and then "help('modules')") or "pydoc modules" in /tmp, but also happens when located anywhere with unreadable subdirs:
ben@Jotunheim:~$ mkdir /tmp/foo; cd /tmp/foo ben@Jotunheim:/tmp/foo$ mkdir bar; sudo chmod 000 bar [sudo] password for ben: ben@Jotunheim:/tmp/foo$ pydoc modules
Please wait a moment while I gather a list of all available modules...
Traceback (most recent call last): File "/usr/bin/pydoc2.6", line 5, in pydoc.cli() File "/usr/lib/python2.6/pydoc.py", line 2309, in cli help.help(arg) File "/usr/lib/python2.6/pydoc.py", line 1765, in help elif request == 'modules': self.listmodules() File "/usr/lib/python2.6/pydoc.py", line 1886, in listmodules ModuleScanner().run(callback, onerror=onerror) File "/usr/lib/python2.6/pydoc.py", line 1937, in run for importer, modname, ispkg in pkgutil.walk_packages(onerror=onerror): File "/usr/lib/python2.6/pkgutil.py", line 105, in walk_packages for importer, name, ispkg in iter_modules(path, prefix): File "/usr/lib/python2.6/pkgutil.py", line 147, in iter_modules for name, ispkg in iter_importer_modules(i, prefix): File "/usr/lib/python2.6/pkgutil.py", line 211, in iter_modules for fn in os.listdir(path): OSError: [Errno 13] Permission denied: './bar'
Proposed patch:
Seems like an easy fix. In Python 3.1.2, change line 206 in /usr/lib/python3.1/pkgutil.py from
if not modname and os.path.isdir(path) and '.' not in fn:
to
if not modname and os.path.isdir(path) and '.' not in fn and os.access(path, os.R_OK):
Other versions much the same (although the specified line number will probably be different.)
Best regards, Ben Okopnik
Author: Ben Okopnik (okopnik)
Date: 2011-04-27 15:43
Here's a test that should exercise every version of "pydoc" installed on the system:
mkdir -p /tmp/foo/bar; cd /tmp/foo; chmod 0 bar
for n in whereis -b pydoc; do echo "**** n∗∗∗∗";n ****"; n∗∗∗∗";n modules; done
Tested under Ubuntu with bash and sh; should work fine with any Bourne-derived shell that supports 'whereis'. Please see attached file.
Author: Éric Araujo (eric.araujo) * 
Date: 2011-04-27 15:49
The script is bugged, since whereis prefixes its output with its argument (i.e. here “pydoc: ”). It’s not a concern anyway: branches open for bugfixes are 2.7, 3.1, 3.2 and 3.3, so when we have a test (preferably as a patch to test_pydoc.py, see http://docs.python.org/devguide for more guidelines on testing and contributing) we check versions and fix where needed.
Author: Ben Okopnik (okopnik)
Date: 2011-04-27 17:27
Trivial fix: please see attached. As to test_pydoc.py, I don't know the system well enough to fiddle with it, but something like this should work (untested):
def test_unreadable_dir(self):
''' pydoc should handle unreadable subdirs gracefully '''
@contextmanager
def mk_unreadable_dir():
top_level_dir = tempfile.mkdtemp()
bad_dir = tempfile.mkdtemp(dir=top_level_dir)
os.chmod(bad_dir, 0)
os.chdir(top_level_dir)
yield
os.removedirs(top_level_dir)
with mk_unreadable_dir():
doc = pydoc.render_doc('modules')
self.assertTrue("modules" in doc)Author: Ben Okopnik (okopnik)
Date: 2011-04-27 21:01
Whoops... with all of that, I just realized that this bug should be filed against pkgutil, not pydoc (pydoc, of course, calls pkgutil to do the path resolution, which is where this crash occurs.) My bad.
import pkgutil inst = pkgutil.ImpImporter(path='/tmp') list(inst.iter_modules()) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.1/pkgutil.py", line 209, in iter_modules for fn in os.listdir(path): OSError: [Errno 13] Permission denied: '/tmp/orbit-gdm'
Author: Ned Deily (ned.deily) *
Date: 2011-10-07 06:08
Thanks for your suggested code and test. Somewhat different fixes to pkgutil have been added by the changes for Issue7367 and a necessary backport of a fix for pydoc from 3.x to 2.7 was made for Issue7425. With these fixes in place, pydoc for 3.3.0, 3.2.3, and 2.7.3 should not exhibit these problems anymore.
History
Date
User
Action
Args
2022-04-11 14:57:16
admin
set
github: 56123
2011-10-07 06:08:33
ned.deily
set
status: open -> closed
resolution: duplicate
messages: +
superseder: pkgutil.walk_packages fails on write-only directory in sys.path
stage: test needed -> resolved
2011-08-06 02:59:32
ned.deily
set
assignee: ned.deily
nosy: + ned.deily
versions: - Python 3.1
2011-04-27 21:01:39
okopnik
set
messages: +
2011-04-27 17:27:44
okopnik
set
files: + pydoc_crash_test
messages: +
2011-04-27 15:49:30
eric.araujo
set
messages: +
2011-04-27 15:43:34
okopnik
set
files: + pydoc_crash_test
messages: +
2011-04-27 08:50:26
eric.araujo
set
nosy: + eric.araujo
stage: test needed
components: + Library (Lib), - Demos and Tools
versions: - Python 2.6, Python 2.5, Python 3.4
2011-04-24 03:34:52
okopnik
create