Issue 29769: pkgutil._iter_file_finder_modules should not be fooled by *.py folders (original) (raw)

The current implementation of _iter_file_finder_modules parses folders with a valid Python module extension as modules (e.g. it would report a folder xy.py as a module xy). As a result, e.g., pydoc.apropos('') fails if such a folder is found anywhere on sys.path.

I'm attaching a patch that fixes this and also brings a few minor improvements (like using a set instead of a dict with 1 values and reusing the function in ImpImporter).

However, I have a question about it (which is also the reason why I didn't turn this into a PR right away): in addition to checking that an item detected as a module is not a directory, I think it would be good to also check that an init module inside a possible package really is a file. If I uncomment the respective check in the patch though, I'm getting a test_pydoc failure because the test creates a package directory with no access to contained file attributes. So even though there is an init.py file in the package dir the isfile() check fails. I think that should, in fact, happen and the pydoc test is wrong, but apparently whoever wrote the test had a different opinion. Any thoughts?

I think there's an open issue somewhere pointing out that _iter_file_finder doesn't handle PEP 420 namespace packages correctly, precisely because it's still looking for an init.py file.

So I'd suggest not worry about changing the init.py handling here, and instead just add a new test case to test_pkgutil that fails with the old traversal code and passes with the new code.