[Python-Dev] a suggestion ... Re: PEP 383 (again) (original) (raw)

Glenn Linderman v+python at g.nevcal.com
Thu Apr 30 09:38:29 CEST 2009

On approximately 4/29/2009 10:17 PM, came the following characters from the keyboard of Martin v. Löwis:

I don't understand the proposal and issues. I see a lot of people claiming that they do, and then spending all their time either talking past each other, or disagreeing. If everyone who claims they understand the issues actually does, why is it so hard to reach a consensus? Because the problem is difficult, and any solution has trade-offs. People disagree on which trade-offs are worse than others.

I'd like to see some real examples of how things can break in the current system Suppose I create a new directory, and run the following script in 3.x: py> open("x","w").close() py> open(b"\xff","w").close() py> os.listdir(".") ['x']

but...

py> os.listdir(b".") ['x', '\xff']

If I quit Python, I can now do

martin at mira:~/work/3k/t$ ls ? x martin at mira:~/work/3k/t$ ls -b \377 x As you can see, there are two files in the current directory, but only one of them is reported by os.listdir. The same happens to command line arguments and environment variables: Python might swallow some of them.

There is presently no solution for command line and environment variables, I guess... which adds some amount of urgency to the implementation of something, even if not this PEP.

and I'd like any potential solution to be made available as a third-party package before it goes into the standard library (if possible). Unfortunately, at least for my solution, this isn't possible. I need to change the implementation of the existing file IO APIs.

Other than initializing them to use UTF-8b instead of UTF-8, and to use the new python-escape handler? I'm sure if I read the code for that, I'd be able to figure out the answer... I don't find any documented way of adding an encoding/decoding handler to the file IO encoding technique, though which lends credence to your statement, but then that could also be an oversight on my part.

One could envision a staged implementation: the addition of the ability to add encoding/decoding handlers to the file IO encoding/decoding process, and the external selection of your new python-escape handler during application startup. That way, the hooks would be in the file system to allow your solution to be used, but not require that it be used; competing solutions using similar technology could be implemented and evaluated.

-- Glenn -- http://nevcal.com/

A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

More information about the Python-Dev mailing list