| msg148258 - (view) |
Author: Éric Araujo (eric.araujo) *  |
Date: 2011-11-24 14:45 |
| I recently changed packaging.util.byte_compile, the function used by the build_py and install_lib commands, so that it can create .pyc and/or .pyo files independently of the calling’s Python -O or -B flags (dad02a080bbc), but I think I introduced a bug. To check that bug and fully test byte_compile before I further simplify the function and backport the whole fix to distutils, I wanted to add tests that load a .pyc or .pyo file and make sure that the right optimization level is used (nothing for --compile, level one for --optimize=1, level 2 for --optimize=2). In the attached patch, I run a Python with -O, -OO or no flag in a subprocess to import a module that prints different things depending on the optimization level. The .pyc or .pyo file is already created by packaging, but I have no check that makes sure that they are not recreated (which I don’t want) and I don’t know if the optimizations in compile.c are applied during byte-compilation or after import (which would make my tests not exercise what I want). (I tried to use imp.load_module at first but it’s not very friendly. Other ideas that I had after the patch was done: Use subprocess + imp so that I can import pyc or pyo as I wish; use open + dis module.) So, could you import experts review the functions added in packaging.tests.support and tell me if they exercise what I want? |
|
|
| msg148260 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2011-11-24 14:57 |
| (For the reference, the bug I added is this: http://hg.python.org/cpython/rev/c10946a17420#l6.45 p7g.util.byte_compile calls py_compile.compile with a filename ending in .pyc or .pyo as appropriate, but the optimization level in the py_compile.compile function depends on the calling Python’s -O option. When byte_compile wants to create a .pyo file but is run under a python without -O, it spawns a subprocess with the -O option. The fix I did made sure that the filename used (pyc or pyo) did not depend on the calling Python’s -O, *but* it should not have removed the setting of -O/-OO in the subprocess, as it’s the only way to control the optimization level in py_compile.compile before 3.2. I think I will use a py_compile or compileall function so that I can have control over the optimization level from Python, which will let me remove the inelegant spawning in byte_compile, and I will backport that function for distutils2. If this makes no sense, please ignore. I’m not sure I would have understood what I’m talking about a year ago.) |
|
|
| msg148437 - (view) |
Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) *  |
Date: 2011-11-27 02:26 |
| What exactly do you consider to backport to distutils? |
|
|
| msg148477 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2011-11-28 13:49 |
| See http://bugs.python.org/issue12119#msg146943 |
|
|
| msg151178 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2012-01-13 15:59 |
| Nick, would you have a bit of time to read my OP and reply? |
|
|
| msg159349 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2012-04-26 01:38 |
| Your basic approach looks sensible to me. One trick I use in test_cmd_line_script to prevent recreation is to simply delete the source file. If the source is gone, implicit recreation is impossible. Unfortunately, that doesn't work for __pycache__, since the cached version will be ignored if the original goes missing. One useful explicit check (as per #14443) would be to ensure the magic number and other attributes are as expected: http://hg.python.org/cpython/file/57d558f1904d/Lib/importlib/_bootstrap.py#l444 You can also pass the "-B" flag to your testing subprocesses, which will switch off the implicit bytecode generation. |
|
|
| msg228510 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2014-10-04 23:29 |
| Looks like someone should revise this patch taking in to account Nick's feedback, and updating it to apply to distutils instead of the now-defunct distutils2/packaging. Moving stage to 'needs patch'. |
|
|
| msg379365 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2020-10-22 21:48 |
| I don’t think tests add value at this stage of the life of distutils; inclined to close this. |
|
|