| msg233719 - (view) |
Author: Franck Michea (franck) * |
Date: 2015-01-09 06:44 |
| Hi, for those of you that prefer to read an example, you can read that commented demonstration of the bug[1]. Today I discovered what I think is a bug in the import system. Here is the basic setup: We have three nested packages: foo -> bar -> baz. The bar package imports foo.bar.baz. We try to import foo.bar. This works well unless we try to alias the foo.bar.baz import in foo.bar with the "import ... as ..." syntax. In that case the file will be read and executed but when we get out of the import statement, then python throws: Traceback (most recent call last): File "", line 1, in File "foo_alias_mod/bar/__init__.py", line 1, in import foo_alias_mod.bar.baz as name_does_not_matter AttributeError: 'module' object has no attribute 'bar' This works whether baz is a package or a module actually. It does not matter if it's from the interpreter, or in a file, ... I've seen it trigger with 2.7.5, 2.7.9, 3.4.5, tip, so I guess this has been here for some time. Please read the link below for a complete demo, and you can always download the tarball[2] to test yourself. [1]: Commented demonstration: http://98810f8c06.net/wtf_python.html [2]: Tarball for test: http://98810f8c06.net/wtf_python-demo.tar.bz2 |
|
|
| msg233744 - (view) |
Author: Alyssa Coghlan (ncoghlan) *  |
Date: 2015-01-09 10:09 |
| You can see the difference between the two cases in the bytecode: >>> dis.dis("import x.y.z") 1 0 LOAD_CONST 0 (0) 3 LOAD_CONST 1 (None) 6 IMPORT_NAME 0 (x.y.z) 9 STORE_NAME 1 (x) 12 LOAD_CONST 1 (None) 15 RETURN_VALUE >>> dis.dis("import x.y.z as bar") 1 0 LOAD_CONST 0 (0) 3 LOAD_CONST 1 (None) 6 IMPORT_NAME 0 (x.y.z) 9 LOAD_ATTR 1 (y) 12 LOAD_ATTR 2 (z) 15 STORE_NAME 3 (bar) 18 LOAD_CONST 1 (None) 21 RETURN_VALUE The aliased version needs to bind the innermost object immediately, so it fails, since "foo.bar" doesn't get set until *after* the import is finished. The version without the alias succeeeds, as it doesn't attempt to eagerly access the attribute before it gets set by the interpreter. To better handle a similar situation with eager attribute lookups during import, issue 17636 changed IMPORT_FROM to fall back to looking at sys.modules when a module attribute it is looking for is missing. Brett, Eric, perhaps it would be worth changing the bytecode emitted in the "import x.y.z as name" case to match that for "from x.y import x as name"? >>> dis.dis("from x.y import z as bar") 1 0 LOAD_CONST 0 (0) 3 LOAD_CONST 1 (('z',)) 6 IMPORT_NAME 0 (x.y) 9 IMPORT_FROM 1 (z) 12 STORE_NAME 2 (bar) 15 POP_TOP 16 LOAD_CONST 2 (None) 19 RETURN_VALUE |
|
|
| msg233768 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2015-01-09 16:51 |
| I think I would need to see exactly how you want the bytecode to change and then think over any backwards-compatibility issues since this is doesn't come up very often. |
|
|
| msg233815 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2015-01-10 11:06 |
| I'm suggesting we change this part of the bytecode emitted for "import x.y.z as bar": 6 IMPORT_NAME 0 (x.y.z) 9 LOAD_ATTR 1 (y) 12 LOAD_ATTR 2 (z) 15 STORE_NAME 3 (bar) to instead emit: 6 IMPORT_NAME 0 (x.y) 9 IMPORT_FROM 1 (z) 12 STORE_NAME 2 (bar) 15 POP_TOP The degenerate case of "import x as y" would be unchanged, only cases which currently emit LOAD_ATTR instructions would be modified. I haven't looked at the practical details yet, but the key would be to use IMPORT_FROM to do the name resolution, rather than LOAD_ATTR, thus enabling the fallback to sys.modules for the eager lookup. |
|
|
| msg233823 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2015-01-10 20:45 |
| That seems reasonable. I guess first step is a patch and then seeing if it passes the test suite. |
|
|
| msg297476 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2017-07-01 06:08 |
| Nick's suggestion was implemented in . |
|
|
| msg314245 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2018-03-22 10:46 |
| Is anything left to do with this issue after ? |
|
|
| msg314249 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2018-03-22 12:24 |
| Heh, apparently I forgot how IMPORT_FROM currently works some time between 2015 and 2017 :) I agree this is out of date now, as the requested behaviour was already implemented for 3.7 |
|
|