[Python-Dev] Unicode strings as filenames (original) (raw)

M.-A. Lemburg mal@lemburg.com
Sun, 06 Jan 2002 17:39:19 +0100

Martin v. Loewis wrote:

We'd still need to support other OSes as well, though, and I don't think that putting all this code into fileobject.c is a good idea -- after all opening files is needed by some other parts of Python as well and may also be useful for extensions.

The stuff isn't in fileobject.c. PyFileSystemDefaultEncoding is defined in bltinmodule.c.

That's the global, sure but the code using it is scattered across fileobject.c and the posix module. I think it would be a good idea to put all this file naming code into some Python/fileapi.c file which then also provides C APIs for extensions to use. These APIs should then take the file name as PyObject* rather than char* to enable them to handle Unicode directly.

Also, on other OSes: You can pass Unicode object to open on all systems. If PyFileSystemDefaultEncoding is NULL, it will fall back to site.encoding.

Of course, if the system has an open function that expects wchart*, we might want to use that instead of going through a codec. Off hand, Win32 seems to be the only system where this might work, and even there, it won't work on Win95.

I expect this to become a standard in the next few years.

I'd suggest to implement something similiar to the DLL loading code which is also implemented as subsystem in Python.

I'd say this is over-designed. It is not that there are ten alternative approaches to doing encodings in file names, and we only support two of them, but it is rather that there are only two, and we support all three of them :-) Also, it is more difficult than threads: for threads, there is a fixed set of API features that need to be represented. Doing PyUNICODE* opening alone is easy, but look at the number of posixmodule functions that all expect file names of some sort.

Doesn't that support the idea of having a small subsystem in Python which exposes the Unicode aware APIs to Python and its extensions ?

-- Marc-Andre Lemburg CEO eGenix.com Software GmbH

Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/