[Python-Dev] PEP 3151 from the BDFOP (original) (raw)

Barry Warsaw barry at python.org
Mon Aug 29 23🔞33 CEST 2011

On Aug 24, 2011, at 01:57 AM, Antoine Pitrou wrote:

One guiding principle for me is that we should keep the abstraction as thin as possible. In particular, I'm concerned about mapping multiple errnos into a single Error. For example both EPIPE and ESHUTDOWN mapping to BrokePipeError, or EACESS or EPERM to PermissionError. I think we should resist this, so that one errno maps to exactly one Error. Where grouping is desired, Python already has mechanisms to deal with that, e.g. superclasses and multiple inheritance. Therefore, I think it would be better to have

+ FileSystemPermissionError + AccessError (EACCES) + PermissionError (EPERM) I'm not sure that's a good idea:

Was it the specific grouping under FileSystemPermissionError that you're objecting to, or the "keep the abstraction thin" principle? Let's say we threw out the idea of FSPE superclass, would you still want to collapse EACCES and EPERM into PermissionError, or would separate exceptions for each be okay? It's still pretty easy to catch both in one except clause, and it won't be too annoying if it's rare.

Yes, FileSystemError might be removed. I thought that it would be useful, in some library routines, to catch all filesystem-related errors indistinctly, but it's not a complete catchall actually (for example, AccessError is outside of the FileSystemError subtree).

Reading your IRC message (sorry, I was afk) it sounds like you think FileSystemError can be removed. I like keeping the hierarchy flat.

Similarly, I think it would be helpful to have the errno name (e.g. ENOENT) in the error message string. That way, it won't get in the way for most code, but would be usefully printed out for uncaught exceptions. Agreed, but I think that's a feature request quite orthogonal from the PEP. The errno number is still printed as it was before:

open("foo") Traceback (most recent call last): File "", line 1, in FileNotFoundError: [Errno 2] No such file or directory: 'foo' (see e.g. http://bugs.python.org/issue12762)

True, but since you're going to be creating a bunch of new exception classes, it should be relatively painless to give them a better str. Thanks for pointing out that bug; I agree with it.

A second guiding principle should be that careful code that works in Python 3.2 must continue to work in Python 3.3 once PEP 3151 is accepted, but also for Python 2 code ported straight to Python 3.3. I don't porting straight to 3.3 would make a difference, especially now that the idea of deprecating old exception names has been abandoned.

Cool.

Do be prepared for complaints about compatibility for careless code though - there's a ton of that out in the wild, and people will always complain with their "working" code breaks due to an upgrade. Be very explicit about this in the release notes and NEWS file, and put your asbestos underoos on. I'll take care about that :)

:)

Have you considered the impact of this PEP on other Python implementations? My hazy memory of Jython tells me that errnos don't really leak into Java and thus Jython much, but what about PyPy and IronPython? E.g. step 1's deprecation strategy seems pretty CPython-centric. Alternative implementations already have to implement errno codes in a way or another if they want to have a chance of running existing code. So I don't think the PEP makes much of a difference for them. But their implementors can give their opinion on this.

Let's give them a little more time to chime in (hopefully, they are reading this thread). We needn't wait too long though.

As for step 1 (coalescing the errors). This makes sense and I'm generally agreeable, but I'm wondering whether it's best to re-use IOError for this rather than introduce a new exception. Not that I can think of a good name for that. I'm just not totally convinced that existing code when upgrading to Python 3.3 won't introduce silent failures. If an existing error is to be re-used for this, I'm torn on whether IOError or OSError is a better choice. Popularity aside, OSError feels more right. I don't have any personal preference. Previous discussions seemed to indicate people preferred IOError. But changing the implementation to OSError would be simple. I agree OSError feels slightly more right, as in more generic.

Thanks for making this change in the PEP.

And that anything raising an exception (e.g. via PyErrSetFromErrno) other than the new ones will raise IOError? I'm not sure I understand the question precisely.

My question mostly was about raising OSError (as the current PEP states) with an errno that does not map to one of the new exceptions. In that case, I don't think there's anything you could raise other than exactly OSError, right?

The errno mapping mechanism is implemented in IOError.new, but it gets called only if the class is exactly IOError, not a subclass:

IOError(errno.EPERM, "foo") PermissionError(1, 'foo') class MyIOError(IOError): pass ... MyIOError(errno.EPERM, "foo") MyIOError(1, 'foo') Using IOError.new is the easiest way to ensure that all code raising IO errors takes advantage of the errno mapping. Otherwise you may get APIs raising the proper subclasses, and other APIs always raising base IOError (it doesn't happen often, but some Python library code raises an IOError with an explicit errno). I also think that rather than transforming exception when raised from Python, i.e. via new hackery, perhaps it should be a ValueError in its own right to raise IOError with an error represented by one of the subclasses. That would make it harder to keep compatibility while adding new subclasses in future Python versions. Imagine a lot of people lobby for a dedicated EBADF subclass and obtain it, then IOError(EBADF, "some message") would suddenly raise a ValueError. Or do I misunderstand your proposal?

Somewhat. FWIW, this is the part that I'm most uncomfortable with.

So, for raising OSError with an errno mapping to one of the subclasses, it appears to break the "explicit is better than implicit" principle, and I think it could lead to hard-to-debug or understand code. You'll look at code that raises OSError, but the exception that gets printed will be one of the subclasses. I'm afraid that if you don't know that this is happening, you're going to think you're going crazy.

The other half is, let's say raising FileNotFoundError with the EEXIST errno. I'm guessing that the init's for the new OSError subclasses will not have an errno attribute, so there's no way you can do that, but the PEP does not discuss this. It probably should.

I found more examples of ECHILD and ESRCH than the former two. How'd you like to add those two to make your BDFOP happy? :) Wow, I didn't know ESRCH. How would you call the respective exceptions? - ChildProcessError for ECHILD?

The Linux wait(2) manpage says:

   ECHILD (for wait()) The calling process does not have any unwaited-for
          children.

   ECHILD (for waitpid() or waitid()) The process specified by pid (wait‐
          pid())  or  idtype and id (waitid()) does not exist or is not a
          child of the calling process.  (This can happen for  one's  own
          child  if  the  action for SIGCHLD is set to SIG_IGN.  See also
          the Linux Notes section about threads.)

- ProcessLookupError for ESRCH?

The Linux kill(2) manpage says:

   ESRCH  The pid or process group does not exist.  Note that an existing
          process might be a zombie, a process  which  already  committed
          termination, but has not yet been wait(2)ed for.

So in a sense, both are lookup errors, though I think it's going too far to multiply inherit from LookupError. Maybe ChildWaitError or ChildLookupError for the former? ProcessLookupError seems good to me.

What if all the errno symbolic names were mapped as attributes on IOError? The only advantage of that would be to eliminate the need to import errno, or for the ugly e.errno == errno.ENOENT stuff. That would then be rewritten as e.errno == IOError.ENOENT. A mild savings to be sure, but still. Hmm, I guess that's explorable as an orthogonal idea.

Cool. How should we capture that?

How dumb/useless/unworkable would it be to add an future to switch from the old hierarchy to the new one? Probably pretty. ;) Well, the hierarchy is built-in, since it's about standard exceptions. Also, you usually get the exception from some library API, so a future in your own module would not achieve much.

What about an api that applications/libraries could use to add additional exceptions based on other errnos they cared about? This could be consulted in PyErrSetFromErrno() and raised instead of IOError. Okay, yeah, that's probably pretty dumb too. The problem is that behaviour becomes inconsistent accross libraries. I'm not sure that's very helpful to the user.

Yeah, on further reflection, let's forget those last two ideas. ;)

Okay, so here's what's still outstanding for me:

Cheers, -Barry

More information about the Python-Dev mailing list