[Python-Dev] File system path encoding on Windows (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sun Aug 21 00:58:46 EDT 2016

On 21 August 2016 at 06:31, Steve Dower <steve.dower at python.org> wrote:

My biggest concern is that it then falls onto users to know how to start Python with that flag.

Not necessarily, as this is one of the areas where commercial redistributors can earn their revenue stream - by deciding that flipping the default behaviour is the right thing to do for their user base (which is inevitably only a subset of the overall Python user base).

Making that possible doesn't mean redistributors will actually follow through, but it's an option worth keeping in mind, as while it does increase the ecosystem complexity in the near term (since default behaviour may vary based on how you obtained your Python runtime), in the longer term it can allow for better informed design decisions at the reference interpreter level. (For business process wonks, it's essentially like running through a deliberate divergence/convergence cycle at the level of the entire language ecosystem: http://theagilepirate.net/archives/1392 )

About the only way I'd be okay with this is going with option #2 (using mbcs:strict, rather than mbcs:replace) and putting a note in the exception to "restart Python with the -X:utf8 option, or contact whoever gave you the application if you don't know what this means", and hope that the user gets to see the exception rather than it being swallowed by the application (and then that the user knows how to start it via python.exe, as opposed to a script entry point, a shortcut or via a browser or other launcher).

Basically, we make breaking noisier and tell the user to fix it. Some users will think that means it is their fault.

I think that makes a good fallback plan if we run into problems during the beta cycle.

On the other hand, having code opt-in or out of the new handling requires changing code (which is presumably not going to happen, or we wouldn't consider keeping the old behaviour and/or letting the user control it), and leads to conflicts between libraries that expect different behaviour. If we expect people to use the flag (and I do), we need to be really explicit that developers should handle two cases. Without the flag, we only need to be explicit about how developers should handle the one case.

Library and framework developers will need to handle both cases anyway, since it's going to be a while before anyone that isn't bundling their language runtime as part of a fully integrated application can assume Python 3.6 as their minimum Python version.

This is a really hard problem for people to reason about abstractly, but "try running Python with this new flag, and see if anything breaks" is a much easier question to ask and answer. Personally I'm a big fan of doing this with beta releases. There's time to switch back to an opt-in model before the final release if things go badly. There's already been a lot of interest in the 3.6 prereleases (the 3.6.0a3 installer had over 80k downloads in its first fortnight), so I think this is viable.

Aye, that sounds like a plausible plan to me.

To help, I could add an extra info page to the prerelease installer announcing some specific new changes and explicitly asking for feedback to bugs.p.o - the ask is already there, but we could beg more in prereleases.

I think you'll want to escalate this to a PEP as well - those aren't just useful as within-development-team design discussion documents, they also help to give changes more visibility to end users during the beta process, and to provide an easily referenced "Why did this change?" explanation for a particular design decision post-release.

That way, "Make it the default during the beta period, potentially revert back to a '-X:utf-8' option in the final beta if we find unresolvable-in-the-near-term problems with changing the default" can be documented as part of the PEP, and folks with concerns about the change will have something to point people to when trying to convince them that they really need to test the 3.6 beta releases on their systems in time to influence that decision.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list