| msg353565 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2019-09-30 11:27 |
| Python path configuration: PYTHONHOME = (not set) PYTHONPATH = (not set) program name = 'D:\a\1\b\layout-appx-amd64\python.exe' isolated = 0 environment = 1 user site = 1 import site = 1 sys._base_executable = 'D:\\a\\1\\b\\layout-appx-amd64\\.\\.\\.\\python.exe' sys.base_prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o' sys.base_exec_prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o' sys.executable = 'D:\\a\\1\\b\\layout-appx-amd64\\.\\.\\.\\python.exe' sys.prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o' sys.exec_prefix = 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o' sys.path = [ 'D:\\a\\1\\b\\layout-appx-amd64\\python38.zip', 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o\\DLLs', 'D:\\a\\1\\b\\layout-appx-amd64\\lib\\o\\lib', 'D:\\a\\1\\b\\layout-appx-amd64\\.\\.\\.', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' |
|
|
| msg353566 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-09-30 12:49 |
| New changeset dec39716ca93ee2e8d9b94915ece33014eb58e9e by Victor Stinner in branch 'master': bpo-38322: Fix gotlandmark() of PC/getpathp.c (GH-16489) https://github.com/python/cpython/commit/dec39716ca93ee2e8d9b94915ece33014eb58e9e |
|
|
| msg353567 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-09-30 12:49 |
| New changeset 18c4ba9f33868761e374a725d497902863d59ea9 by Victor Stinner in branch '3.8': bpo-38322: Fix gotlandmark() of PC/getpathp.c (GH-16490) https://github.com/python/cpython/commit/18c4ba9f33868761e374a725d497902863d59ea9 |
|
|
| msg353572 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-09-30 13:34 |
| Facts: * appx tests are not run on buildbots * appx tests are not run in pre-commit CI of pull requests * appx tests seem to only be run on Azure Pipelines of the branches (like 3.8 and master) * the fixed bug likely existed since Python 3.6: static int gotlandmark(const wchar_t *landmark) { Py_ssize_t n = wcsnlen_s(prefix, MAXPATHLEN); join(prefix, landmark); ... prefix[n] = '\0'; ... } with join which uses _PathCchCombineEx() or PathCombineW() ... No idea why the bug decided to show up today, 3.8.0rc1 release day. Anyway, the Azure Pipelines CI is back to green on the 3.8 branch. |
|
|
| msg353582 - (view) |
Author: Steve Dower (steve.dower) * |
Date: 2019-09-30 14:52 |
| The bug was probably exposed due to a change in path initialization (there's been a few changes here recently ;) ) that stopped normalising prefix. You can see that sys.executable in your dumped path has additional "." in the path, and that's never been the case before - they're always cleaned up earlier than that. Adding in the appx _layout_ test (it's the whole test suite in an "installed" layout) was easier to do in CI than creating a buildbot config, so I threw it in as a test and then left it on. It's certainly not been failing this whole time. |
|
|
| msg353614 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-09-30 19:46 |
| > The bug was probably exposed due to a change in path initialization (there's been a few changes here recently ;) ) that stopped normalising prefix. You can see that sys.executable in your dumped path has additional "." in the path, and that's never been the case before - they're always cleaned up earlier than that. Yeah, I had to modify a lot of "path config" code to fix many bugs (regressions compared to Python 3.7). I also added the first tests on the "path configuration" in test_embed. I checked why sys.executable is not normalized in the the "appx build". PC\python_uwp.cpp calls set_process_name() which sets PyConfig.executable and the path is not normalized. If PyConfig.executable is set explicitly, the path is left unchanged: PyConfig has the highest priority. Should the executable path be normalized in this case? -- I looked at Python 3.6 (the version before I started to rework the Python initialization). But there is no python_uwp.cpp in this version. In Python 3.7, python_uwp.cpp calls set_process_name() which calls _Py_SetProgramFullPath(). But config_init_path_config() seems to ignore the _Py_SetProgramFullPath() call in Python 3.7. |
|
|
| msg353615 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2019-09-30 19:47 |
| > Adding in the appx _layout_ test (it's the whole test suite in an "installed" layout) was easier to do in CI than creating a buildbot config, so I threw it in as a test and then left it on. It's certainly not been failing this whole time. I'm fine with keeping it here. I don't pay much attention to it, since I don't get an email when a Azure Pipelones job fails, and the UI is kind of "hidden" in GitHub (not as visible as CI results on a PR). |
|
|
| msg353618 - (view) |
Author: Steve Dower (steve.dower) * |
Date: 2019-09-30 20:27 |
| > I checked why sys.executable is not normalized in the the "appx build". PC\python_uwp.cpp calls set_process_name() which sets PyConfig.executable and the path is not normalized. If PyConfig.executable is set explicitly, the path is left unchanged: PyConfig has the highest priority. Ah, that would explain it, certainly. I don't know why the dots are ever added though. This test also has the weird oddity that it isn't running as an actual app, so a different initialization path is taken. In this case, it should be using the final name as reported by the operating system, so the extra dots are basically unexplainable. I'll try a couple of experiments and see if I can figure it out. > the UI is kind of "hidden" in GitHub Yeah, but I look at it periodically (the "Commits" view is a convenient way to see recent CI and buildbot status). It's considerably simpler for me than finding out it's broken in an RC :) |
|
|
| msg353619 - (view) |
Author: Steve Dower (steve.dower) * |
Date: 2019-09-30 20:31 |
| Either way, the paths in the latest tests [1] seem to be fine, and they all passed, so I don't think this is a release blocker anymore. Thanks, Victor! [1]: https://dev.azure.com/Python/cpython/_build/results?buildId=52024&view=logs&j=c8a71634-e5ec-54a0-3958-760f4148b765 |
|
|