[Python-Dev] Using logging in the stdlib and its unit tests (original) (raw)

Glenn Linderman v+python at g.nevcal.com
Sat Dec 11 10:30:58 CET 2010

On 12/11/2010 12:00 AM, Nick Coghlan wrote:

On Sat, Dec 11, 2010 at 4:25 PM, Glenn Linderman<v+python at g.nevcal.com> wrote:

On 12/10/2010 9:24 PM, Nick Coghlan wrote: This could actually make a reasonably good basic for a "task oriented" subsection of the logging documentation. Something like:

Yep, agree. But sadly, for each point, there may be multiple options (your StreamHandler, but I'd want a FileHandler; your separation of messages by level, my wanting them combined; etc.) No, no, no, that's the whole point of using logging. The library writer doesn't really care about where the messages end up - that is entirely in the hands of the application developer when they choose which handlers to install. The only situation that the library writer cares about is the one that tripped up concurrent.futures and that has already been changed for 3.2b2: that warnings and errors written to a logger were silenced by default if the application never even called basicConfig(). That has now been fixed so they will be emitted on sys.stderr instead (including obeying any redirections of stderr over the lifetime of the program). Notice that my task list is entirely from the point of view of the person emitting the messages. How those messages are later displayed is then up to the application writer (or the logging module default settings, if the application writer doesn't do anything specific).

Yes, yes, yes... that's what your list was! I see that now! But it is interesting that in missing that point, and reading your logging setup code, I saw that the same list of tasks is also exactly the sorts of things that you would want to control, for simple uses of the logger.
Does basicConfig give that control? I don't know, the first 8% of the logger documentation don't tell me that, they only give me a canned example (actually 3) without explanation of its full function. Someday I'll read more. Until then, basicConfig is opaque: I know only three incantations for using it, and not even any surety that they can be mixed, or that it can be called more than once in the same program.

Your comment about basicConfig setting the level on the root logger, but not on the default handler making it useless is opaque to me, An early version of my code used basicConfig to create the stderr StreamHandler, but that turned out to be pointless since I wanted different levels on the logger and the handler.

but is there perhaps room for another basic setup API that could get the setup code down to a line or two in simple cases? For 3.2? No. For 3.3? Maybe. Would that be a useful set of functionality to bundle? And could it be extended, when the user wants more power, or would it have to be replaced, because it gets in the way of the user that wants more power? Logging already has powerful configuration mechanisms, especially following the addition of dictConfig (http://docs.python.org/dev/library/logging#logging-config-dictschema), so it really doesn't need anything else along those lines.

Sure; that is the replacement technique. And maybe that is OK, if there are no useful intermediates between basicConfig (whatever it might be capable of) and dictConfig (or the other various config techniques that preceded it and still exist, if I understand correctly from has been said here on python-dev).

At the simpler end, basicConfig already covers sending all messages to a single stream with a single format - it's only split-stream handling (such as stderr/stdout potentially being different endpoints) that it can't cope with.

OK, so I'd probably be fine with basicConfig, even though you weren't, once I understand it a little better.

What may make more sense than yet another global config mechanism, is a module level "addHandler" helper function along the following lines:

And once I learn enough to understand the addHandler code, I will have read a lot of the logger documentation than I have to date, or it will have been restructured enough that re-reading the first 8%, or maybe 12% if the easy-to-understand part grows! But I like what you said here:

# Previous set up example is now only three lines long import sys, logging # Let root logger handlers see all messages logging.getLogger().setLevel(logging.NOTSET) # Send WARNING and above to stderr addHandler(stream=sys.stderr, level=logging.WARNING) # Send INFO to stdout addHandler(stream=sys.stdout, level=logging.INFO, maxlevel=logging.INFO)

logging.info("Hello world!") logging.warn("Hello world!")

And this subset usage of addHandler is reasonably understandable for beginners, although clearly the complexity of all the parameters must add power that is not needed in the simple case, that you feel could be useful in the complex case. And I would change the third line from level=logging.INFO to level=logging.DEBUG to see lots more on stdout, correct?

Looking back at the logger's simple examples section, I see that getLogger is used before being defined, is not explained, nor is its parameter explained. So it is not clear how to go from the first simple example to the second simple example. I wonder if that is where I started getting a bit of uncertainty about the ability to quickly learn what was going on... the rotating log example uses several APIs that haven't been used before, without explaining them at all. Can that example actually be added to the previous example? What is the magic 'MyLogger'? Where has basicConfig() gone in the second example?

I'm not expecting answers to the above questions here, but I'd love to see the documentation improved to the point that I don't have them any more :)

Glenn -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20101211/05c86a1b/attachment.html>

More information about the Python-Dev mailing list