| msg258509 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2016-01-18 09:10 |
| Attached patch implements the largest part of the PEP 511: * add sys.get_code_transformers() and sys.set_code_transformers() * add sys.implementation.optim_tag * add -o command line option setting sys.implementation.optim_tag * modify importlib to use sys.implementation.optim_tag to build .pyc filenames * modify importlib to raise an ImportError when the .pyc file is missing, the .py is available, and the optimizer tag created from code transformers is different than the current sys.implementation.optim_tag * add Lib/test/test_pep511.py: test changes especially bytecode and AST transformers * add a PeepholeOptimizer object created by default for sys.set_code_transformers() * skip the peephole optimizer when -o noopt is used * add sys field to the PyInterpreterState structure: used by _PySys_GetCodeTransformers() to get code transformers * update unit tests to use the optimizer to create the .pyc filename An optimizer tag must contain characters: '.', '-', directory separators or NUL character. Brett Canon proposed to not include the optimizer tag in .pyc filenames if it is "opt" to limit backward incompatible changes. Note: The PEP 511 is under discussion on python-ideas. The patch implements the first version of the PEP, sent to python-ideas, so it doesn't include proposed changes like changing code_transformer() prototype to "def code_transformer(self, code, context)". See also the issue #26100: add test.support.optim_args_from_interpreter_flags(). |
|
|
| msg258511 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-18 09:15 |
| Missing patch for test_sys testing the new sys.get/set_code_transformers() functions. |
|
|
| msg258560 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-18 23:40 |
| main.c: + case 'o': + if (wcslen(_PyOS_optarg) == 0 + | |
wcschr(_PyOS_optarg, L'.') + |
|
| msg258573 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-19 07:44 |
| sys_init_code_transformer: Py_DECREF(name); must be Py_XDECREF(name); at exit, it can NULL. An unit test on an invalid transformer (with no name) is missing? |
|
|
| msg258580 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-19 10:49 |
| Patch version 2: * allow "-" character in the argument of the -o command line option * add an unit test on invalid type for code transformer * fix a crash in sys.set_code_transformers(): replace Py_DECREF() with Py_XDECREF() |
|
|
| msg258581 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-19 10:51 |
| (oops, i posted the same message with patch twice) |
|
|
| msg258678 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-20 12:41 |
| Note: ast.PyCF_TRANSFORMED_AST is not implemented yet. |
|
|
| msg258822 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-22 16:33 |
| Rebased patch. |
|
|
| msg258829 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-22 16:48 |
| Patch version 4: Fix sys_set_code_transformers(), initialize seq to NULL to fix a crash on error handling. |
|
|
| msg258866 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-01-23 12:11 |
| > Note: ast.PyCF_TRANSFORMED_AST is not implemented yet. Additionnal pycf_transformed_ast.patch implements it. The patch should be applied on top of transformers-4.patch. Note: PyCF_TRANSFORMED_AST has the same value (0x1000) than the old constant CO_GENERATOR_ALLOWED. This constant was removed in Python 2.5.0 by the changeset 6b42920accc9 in 2006. Is it an issue? |
|
|
| msg259648 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-02-05 09:15 |
| Rebased patch combining also pycf_transformed_ast.patch. |
|
|
| msg260217 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-02-13 00:30 |
| Patch version 6: updated to the new code_transformer() API, it now takes a code object as input and must return a new code object. Sadly, it looks like there are reference leaks (try: ./python -m test -R 3:3 test_import). |
|
|
| msg260222 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2016-02-13 01:07 |
| TODO: implement a fast-path for PyCode_Optimize() avoiding completly the "expensive" code transformer API, before the first call to sys.set_code_transformer(). It would avoid list <=> tuple conversions for code constants, and the need of creating two code objects. In practice, this fast-path should be taken in most cases. |
|
|
| msg296704 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-06-23 12:29 |
| Recently, some people asked me for an update for my FAT Python project. So I rebased this change I wrote 1 year 1/2 and adapted it for the new code base: * I renamed test_pep511.py to test_code_transformer.py * I removed the sys module from the PyInterpreterState structure (added in my previous patch), since Eric Snow and Nick Coghlan removed a variant of the sys module from this structure for their work on subinterpreters and reworked Python initialization The PEP 511 is not accepted, so the implementation is still a work-in-progress (WIP) and must not be merged. |
|
|
| msg304529 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-10-17 20:25 |
| I rejected my own PEP 511, so I now reject this issue as well. https://mail.python.org/pipermail/python-dev/2017-October/149903.html |
|
|