[Python-Dev] Using logging in the stdlib and its unit tests (original) (raw)

Paul Moore p.f.moore at gmail.com
Wed Dec 8 14:02:00 CET 2010

On 8 December 2010 08:32, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

Nick Coghlan <ncoghlan gmail.com> writes:

I'm not proposing that the standard library be special-cased, I'm proposing that the default behaviour of an unconfigured logging module in general be changed to something more useful (i.e. warnings and errors printed to stderr, everything else suppressed), rather than unhelpfully suppressing all output except for an "Oh, I'm throwing output away" message the first time it happens. I don't have a philosophical problem with this, but there might be differing opinions about this not just on python-dev but also in the wider community. I'd definitely want logging to do the right thing and minimize inconvenience as much as possible. There's also possibly a backwards compatibility dimension to this, but I'm not sure to what extent a change like this would really affect people in practice (as nothing will change if logging is configured).

To provide an application developer point of view, I'm +1 on this change.

I want library developers to be able to report errors/warnings and have them appear by default, without me doing anything (or even needing to be aware that the logging module is involved). The benefit of the library developer using the logging module to me is that if I deem it appropriate, I can direct the warnings/errors elsewhere simply by configuring logging.

So, I'd like the library developer to feel that they can use logging in precisely the same ways as they would write to sys.stderr, and with the same expectations (output will be seen by default), but with less reluctance because they know that I can easily adapt their output according to my needs.

The options for handling this are:

1. Write the error detail directly to stderr. (Unless the default behaviour of logging changes, that is what I am going to suggest Brian do, as it exactly mimics the behaviour of the PyErrWriteUnraisable API).

I find library code that does this annoying. It's the right effect, but leaves me with little or no control should I want to do something like log to a file.

2. Write it to the root logger with the convenience APIs (Possible option, but I don't like the global state impact of implicitly calling basicConfig() from library code)

Library code shouldn't use the root logger, and shouldn't call basicConfig - I agree with Nick again.

3. Add a StdErr handler for the logger (this is what is currently implemented, and again, I don't like it because of the global state impact on something that should be completely under an application's control)

That seems like a best of a bad job answer. As an application user, I'd expect to find it awkward to modify this default, purely because I have to undo the configuration that the library had to do to work around the logging library's defaults.

Thanks for the detailed explanation. I agree that unraisable warnings and errors need to be handled somehow. There is a way in which this can be done without affecting a logging configuration, viz. logging can define a "handler of last resort" (not attached to any logger) which is invoked when there are no user-specified handlers. This would by default be a StreamHandler writing to sys.stderr with a threshold of WARNING (or perhaps ERROR). Thus sounds like a better option than a direct write to sys.stderr, since you can't change the latter behaviour easily if you want to do something else instead.

From my POV as an application writer, I don't care how you implement it (but I agree that your suggestion sounds more flexible). As long as errors and warnings are printed by default, and I can set my own configuration without having to undo the existing configuration, then I'm happy.

This is of course a backwards-incompatible change to logging semantics: instead of saying that logging will be silent unless explicitly asked to produce output, we're saying that logging will always produce output for warnings and errors (or perhaps just errors), unless explicitly silenced. This is of course in line with the Zen of Python; the present behaviour, which is not so aligned, is based on the idea that logging should not affect program behaviour if it's not wanted by the program developer (as opposed to library developer).

I can see that it's an incompatible change. But I'd still be in favour of it for 3.2. In my view, it needs to be done, and it'll only get harder to do as time goes on.

Paul.

More information about the Python-Dev mailing list