[Python-Dev] Pathlib enhancements - acceptable inputs and outputs for fspath and os.fspath() (original) (raw)

Brett Cannon brett at python.org
Tue Apr 19 12:50:07 EDT 2016

On Tue, 19 Apr 2016 at 04:46 Stephen J. Turnbull <stephen at xemacs.org> wrote:

Brett Cannon writes: > On Mon, 18 Apr 2016 at 12:26 Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Well, it makes your head hurt; It doesn't, because I have a different (and IMHO better) model. I can interpret yours without pain by comparing to that. > By providing os.fspath() I can say that I do not, under any > circumstances, want someone to guess at the encoding some bytes > path is under to get me a string and instead I want to start and > end entirely in a world of strings. IOW os.fspath() lets me work in > such a way that the instant bytes are introduced into my code for > file paths it triggers a TypeError. Does it really help you work that way? open is polymorphic, and will use os.rawfspath(obj, (bytes,str)). Ditto os.scandir etc. If they don't, there's no point in supporting bytes returns from fspath, is there?

You're leaving out all of the os.path functions, but you're right that if they didn't support it like Windows then this entire discussion of bytes paths would be moot.

Application code will normally not be calling os.fspath. In the future, pathlib will, I suppose, but even without os.fspath pathlib already protects you, as does antipathy.[1]

I disagree that application code won't be calling os.fspath.

More effective, then, is just to use pathlib for your Path-hacking work as soon as the path-representing object appears, and Path will complain about bytes for you. This is an analogue of the "decode bytes at the boundary" principle.

Ah, but you see that doesn't make porting easy. If I have a bunch of path-manipulating code using os.path already and I want to add support for pathlib I can either (a) rewrite all of that path-manipulating code to work using pathlib, or (b) simply call path = os.fspath(path) and be done with it. Basically if you have written any code that uses os.path then you will have to care about (a) or (b) as a way to add support for pathlib short of the str(path) hack we're all working to get away from. And if people truly liked option (a) then this conversation wouldn't be such a big deal as we would have seen more people using pathlib already (yes, the provisional tag may have scared some off, but my guess is it's more from not wanting to rewrite os.path-using code).

Now if you can convince me that the use of bytes paths is very minimal and thus people doing path manipulations with them will be a very small minority then I'm happy to try and use this to keep pushing people towards avoiding bytes for file paths. But over the years people such as yourself, Stephen, have convinced me that people do some really crazy stuff with their file systems and that it isn't isolated to just one or two people. And so it becomes this situation where we need to ask ourselves if we are going to tell them to just deal with it or help them transition.

The other way to convince me is that people needing to support older versions of Python will use path = path.__fspath__() if hasattr(path, '__fspath__') else path and that allowing bytes with that idiom is going to cost them dearly. My current assumption is that it won't because people using that idiom are using os.path and those functions will complain when mixing str and bytes together, but I'm open to being convinced otherwise.

I guess what I'm trying to get at is that I understand the desire to get people to get the bytes path habit, but to me the best way will be to get people quickly and easily transitioned over to pathlib as a carrot rather than using the lack of bytes path support in this transition as a stick.

-Brett

> Yep, we are stuck with the names unless you want to propose a new > name and deprecate the old one. I already proposed fsensurebytes and fsensurestr. I think they're sufficiently ugly to prove my point.

Footnotes: [1] Strictly speaking, antipathy protects you from inadvertant mixing of bytes and str. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20160419/531d2be8/attachment.html>

More information about the Python-Dev mailing list