[Python-Dev] Using logging in the stdlib and its unit tests (original) (raw)

Vinay Sajip vinay_sajip at yahoo.co.uk
Wed Dec 8 18:43:06 CET 2010

Robert Kern <robert.kern gmail.com> writes:

I'm sorry, but it's not at all clear that you have understood my point. There is no way for me to parse your words as a sensible reply to what I said.

Let's say I write a library called Foo. I want to add logging to my functions. You want to write an application called Bar that uses Foo and you want to configure logging for your application (at the very least to provide a default if not production). The warning is supposed to help you not make mistakes when configuring logging in your application. If I, library author, attach NullHandlers to all of Foo's loggers, then you will not get that warning if you forget to add handlers the Foo loggers. My adding the NullHandler to Foo prevented that warning that you consider to be so important.

I understand, and what you say above is correct.

I don't think the warning helps much, if at all.

Perhaps not. I'm not so hung up on the importance of the message now, but it certainly was important when logging was first introduced, and users couldn't expect to get the same level of help on comp.lang.python that they now can. Today there are quite a few people who can help people with finger-trouble logging issues.

Consider the scenarios when not having the current behaviour would bite you:

  1. You're new to Python/new to logging. You write some code, perhaps across several modules, which code makes logging calls, but you omit to configure any handlers, whether through forgetting or not understanding what needs to be done. Without the message, you've no idea why no logging messages appear, no matter how much you fiddle with levels.
  2. You write some code and decide you don't want to use logging, but use some third party libraries which do. You don't care about that, so it's annoying to have "no handlers could be found for logger XXX" messages printed to console. You berate the library developer for their lack of consideration.

Perhaps you don't find yourself in these situations, but surely you sympathize with people who do? How would you propose to address both those scenarios?

In a large package (particularly a namespace package), I can't guarantee that any particular module will get imported. I will want to be able to import just foo.bar.baz without needing to worry about whether foo.setuplogging got imported and ran the logging configuration as a side-effect. I want to be able to loosen the coupling between modules across my package, not add more coupling.

I'm not sure what coupling you're talking about - perhaps you can illustrate with an example. If I develop a package "foo.bar" which is part of namespace package "foo", and use loggers named name in my code, and add a NullHandler to logger "foo.bar", that's all I have to do. Likewise, if another person develops "foo.baz" and they add a NullHandler to "foo.baz", then where's the coupling between the two packages? They needn't even know about each other.

But in any case, while adding a NullHandler to just the package's root logger helps you to avoid needing a NullHandler on every logger, the effect is the same. Almost all loggers effectively terminate in a NullHandler either directly or through a chain of parent loggers. So why not just make it the default?

There's no "termination" when a NullHandler is encountered. Django has dozens of modules, many of which could use logging, but only one NullHandler needs to be added for the whole of Django. The effect you say is the same is not: when adding new modules to Django, for example, no additional NullHandler adding needs to be done.

I don't want to appear rude, but this seems to be another mistake (or perhaps a variant of the same mistake as before) you are making about how logging works. You obviously feel strongly about it, and if you have a specific use case which is causing/has caused you pain, please spell it out for me (on comp.lang.python, say) and I will try to help sort out the problem.

As am off-topic example, Armin Ronacher kept on saying in various posts and presentations that you couldn't use stdlib logging for web applications, that there were fundamental problems with it. But when he actually sent me his specific problem statement, I gave him a solution without spending too much time on it (see http://plumberjack.blogspot.com/2010/09/configuring-logging-for-web.html if you care). I'm not trying to be obstructive, honestly.

Personally, I would back the proposals being made elsewhere in this thread, that in the absence of configuration, warnings and errors should be printed to stderr no matter where they come from. This gives useful behavior out-of-the-box

As you might have seen in my response to Nick's post, I've made specific proposals about this myself. I agree with Nick's viee that some more time should be given for discussion, suggestions, others to express views, and just for the ideas to bake for a bit.

without configuration but remains completely configurable. Library errors don't pass silently, but logging allows people to silence them explicitly. It separates the concerns of library authors (who should never touch logging configuration and shouldn't be required to think about it) from those of the application authors and application users.

You surprise me with "library authors (who should never touch logging configuration and shouldn't be required to think about it)". If you use logging, you should think about it at least a little, just as with any other software you use. For example, about default levels, propagation, etc. etc. The Django team, for example, have expended a reasonable amount of thought on how best to add logging support to their library; I don't believe their time has been wasted.

Regards,

Vinay Sajip

More information about the Python-Dev mailing list