Issue 13591: import_module potentially imports a module twice (original) (raw)
Created on 2011-12-12 21:43 by Ryan.Twitchell, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (9)
Author: Ryan Twitchell (Ryan.Twitchell)
Date: 2011-12-12 21:43
Use of importlib's import_module function with modules belonging to a library can cause some modules to be imported twice, if such a module is referenced from sibling modules, and from init in the package. I suspect this is a bug, or at best a nuance of packages that is quite subtle.
Easier to show with an example. Below, the module my_lib.bar will be imported twice. Given the following file structure:
./scratch.py ./my_lib/init.py ./my_lib/foo.py ./my_lib/bar.py
And the following file contents:
./scratch.py:
import importlib
importlib.import_module('my_lib.bar')
importlib.import_module('my_lib.foo')./my_lib/init.py:
import my_lib.bar
# Or alternately
#from . import bar./my_lib/foo.py:
from . import bar
print('In foo, id(bar.baz): %s' % id(bar.baz))./my_lib/bar.py:
def baz():
pass
print('In bar, id(bar.baz): %s' % id(baz))Running scratch.py results in my_lib.bar being imported twice: $ echo $PYTHONPATH . $ python --version Python 3.2.2 $ python ./scratch.py In bar, id(bar.baz): 21328632 In bar, id(bar.baz): 21352240 In foo, id(bar.baz): 21352240
Replacing the calls to import_module with use of the import statement, or import, or simply rearranging the order of the two calls all result in the module my_lib.bar to be imported only once. As does eliminating the import statement in my_lib.init.
This may be a misunderstanding on my part regarding the intended use of packages, but this behavior was quite unexpected, and rather difficult to track down in real code.
Author: Alyssa Coghlan (ncoghlan) * 
Date: 2011-12-13 02:25
At first glance, I thought this might be just the circular import problem (#992389) appearing in a different guise. However, if that was the case, switching to an import statement or import shouldn't have made any difference.
What do you see if you put an "import my_lib.bar" between the two importlib calls?
Author: Meador Inge (meador.inge) *
Date: 2011-12-13 15:17
I can reproduce this on tip. What happens is that 'importlib.import_module("my_lib.bar")' is effectively computed as:
import my_lib import bar
by '_bootstrap._gcd_import'. When '_gcd_import' goes to do the import of 'bar' it does not check to see if 'bar' has already been imported by the parent import.
Here is a patch without tests that fixes this. I will add the tests next.
Author: Ryan Twitchell (Ryan.Twitchell)
Date: 2011-12-14 04:18
Confirmed that this patch fixes the behavior shown in my original example, with 3.2.
Author: Meador Inge (meador.inge) *
Date: 2011-12-14 05:45
Updated patch with tests.
Author: Brett Cannon (brett.cannon) *
Date: 2011-12-14 21:29
Patch looks good to me.
Author: Roundup Robot (python-dev) 
Date: 2011-12-15 04:28
New changeset d2504d30f259 by Meador Inge in branch '3.2': Issue #13591: import_module potentially imports a module twice. http://hg.python.org/cpython/rev/d2504d30f259
New changeset e8fb61a0a2d7 by Meador Inge in branch 'default': Issue #13591: import_module potentially imports a module twice. http://hg.python.org/cpython/rev/e8fb61a0a2d7
Author: Meador Inge (meador.inge) *
Date: 2011-12-15 04:30
Thanks for the review Brett. Fix committed.
Author: Roundup Robot (python-dev)
Date: 2011-12-15 04:41
New changeset 541f215a31f7 by Meador Inge in branch '3.2': Issue #13591: Moving the NEWS line to the right release. http://hg.python.org/cpython/rev/541f215a31f7
New changeset 92e94fd303d4 by Meador Inge in branch 'default': Issue #13591: Moving the NEWS line to the right release. http://hg.python.org/cpython/rev/92e94fd303d4
History
Date
User
Action
Args
2022-04-11 14:57:24
admin
set
github: 57800
2011-12-15 04:41:12
python-dev
set
messages: +
2011-12-15 04:30:14
meador.inge
set
status: open -> closed
resolution: fixed
messages: +
stage: commit review -> resolved
2011-12-15 04:28:05
python-dev
set
nosy: + python-dev
messages: +
2011-12-14 21:29:11
brett.cannon
set
assignee: meador.inge
messages: +
stage: patch review -> commit review
2011-12-14 05:45:04
meador.inge
set
files: + issue13591-2.patch
messages: +
2011-12-14 04🔞08
Ryan.Twitchell
set
messages: +
2011-12-13 15:46:37
pitrou
set
versions: + Python 3.3
2011-12-13 15:17:19
meador.inge
set
files: + issue13591.patch
keywords: + patch
messages: +
stage: patch review
2011-12-13 02:25:15
ncoghlan
set
messages: +
2011-12-13 01:53:46
meador.inge
set
nosy: + meador.inge
2011-12-12 21:44:50
ezio.melotti
set
nosy: + brett.cannon, ncoghlan
2011-12-12 21:43:23
Ryan.Twitchell
create