Issue 14209: pkgutil.iter_zipimport_modules ignores the prefix parameter for packages (original) (raw)

Created on 2012-03-06 15:02 by James.Pickering, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (7)

msg155018 - (view)

Author: James Pickering (James.Pickering)

Date: 2012-03-06 15:02

If you run pkgutil.iter_zipimport_modules with a prefix parameter, and the module in question is a package, then the prefix parameter is ignored.

The most visible symptom of this is when running pkgutil.walk_packages for a zipfile. Imagine we have a module structure like this (or create one):

a/ a/init.py a/b/ a/b/__init.__py

If we put this structure in a directory, add the directory to sys.path, and run pkgutil.walk_packages(), it will find modules "a" and "a.b".

If we put this structure in a zipfile, however, we add this file to sys.path, and run pkgutil.walk_packages(), it will find modules "a" and "b". This is because pkgutil.iter_zipimport_modules ignores the prefix parameter "a.".

This is incorrect.

This can be fixed by changing line ~344 of Lib/pkgutil.py from:

yield fn[0], True

to

yield prefix + fn[0], True

Thanks, James

P.s, This is my first Python bug report. I apologise in advance for any poor etiquette.

msg155137 - (view)

Author: Alyssa Coghlan (ncoghlan) *

Date: 2012-03-08 00:15

Adding Brett, since the plan is to clear out a lot of the redundant code in pkgutil once importlib is fully bootstrapped as the standard import implementation.

(although this will still affect the older versions directly)

msg265221 - (view)

Author: Łukasz Langa (lukasz.langa) *

Date: 2016-05-10 03:06

Brett, Facebook is using the proposed patch in prod since last year. It works fine. Maybe we should just include it for the time being?

msg265223 - (view)

Author: Brett Cannon (brett.cannon) *

Date: 2016-05-10 03:09

I don't deal with pkgutil so I have no problem if you want to go ahead and apply the patch if you think it's reasonable to accept, Łukasz.

msg265241 - (view)

Author: Alyssa Coghlan (ncoghlan) *

Date: 2016-05-10 11:49

+1 for just including the fix in the next round of maintenance releases, although a test case would be desirable.

msg268306 - (view)

Author: Roundup Robot (python-dev)

Date: 2016-06-12 01:07

New changeset 9649acf7d472 by Łukasz Langa in branch '3.5': Issue #14209: pkgutil.iter_zipimport_modules ignores the prefix for packages https://hg.python.org/cpython/rev/9649acf7d472

New changeset 389b7456a053 by Łukasz Langa in branch 'default': Merge 3.5, issue #14209 https://hg.python.org/cpython/rev/389b7456a053

msg268307 - (view)

Author: Łukasz Langa (lukasz.langa) *

Date: 2016-06-12 01:09

Done. Fix is going to be present for 3.5.2 and 3.6. Thank you, James.

History

Date

User

Action

Args

2022-04-11 14:57:27

admin

set

github: 58417

2016-06-12 01:09:36

lukasz.langa

set

status: open -> closed
resolution: fixed
messages: +

versions: + Python 3.5, Python 3.6, - Python 2.7, Python 3.2, Python 3.3

2016-06-12 01:07:40

python-dev

set

nosy: + python-dev
messages: +

2016-05-10 17:44:09

brett.cannon

set

assignee: lukasz.langa

2016-05-10 11:49:16

ncoghlan

set

messages: +

2016-05-10 03:09:45

brett.cannon

set

messages: +

2016-05-10 03:06:15

lukasz.langa

set

nosy: + lukasz.langa
messages: +

2012-03-09 08:54:28

eric.snow

set

nosy: + eric.snow

2012-03-08 00:15:20

ncoghlan

set

nosy: + brett.cannon
messages: +

2012-03-07 22:06:45

eric.araujo

set

nosy: + pje, ncoghlan

stage: test needed

2012-03-06 15:02:24

James.Pickering

create