[Python-Dev] Consistent logging in the standard library (original) (raw)

Guido van Rossum guido at python.org
Wed Sep 10 00:15:37 EDT 2003

The method for doing so is modelled after the socket module's new timeout functionality [3], where each module that performs logging defines a pair of module-level functions named getdefaultlogger() and setdefaultlogger() that return and accept (respectively) a logging.Logger instance. The initial default logger for each of these modules is named after the module itself (i.e. logging.getLogger(name)) and its behavior mimics that of the Python 2.3 version of the module in terms of format and destination of the logging messages.

Hm... Isn't the logging module's ability to set separate handlers sufficient to get the desired flexibility? Rather than the application specifying a different logger object, the application can specify a different handler object. I think this was the originally intended design. It places less of a burden on the modules that use logging (no need to define getdefaultlogger() and setdefaultlogger()) and makes it possible for the different handlers to be specified by the logging module's configuration file, without the application having to support this.

Then what remains is a convention for the name of the logger used by standard library modules; perhaps the module name should suffice?

So an application that wants to override the logging behavior of a standard library module can do so simply by passing a customized logger object to the module's setdefaultlogger() function. This works out even better for modules where all the logging is encapsulated in a class.

The PEP calls for each class that performs logging to define an attribute called self.logger, and upon instantiation initialize self.logger to the module's current default logger (again, modelled after how the timeout value for a socket object is initialized). This allows each instance of a class to have its own unique logging behavior.

What behavior exactly are you thinking of? A logger object has almost no semantics apart from passing log events on to handlers; by design, all the work is done in the handlers.

The first item for discussion is whether this seems like a reasonable approach. The second is identifying which standard library modules are candidates for modification using this approach. I've identified the following by simply searching for the word "logging" in the source code, but I'm not sure whether this list is complete:

asyncore BaseHTTPServer cgi doctest imaplib unittest the distutils package I have patches ready for asyncore and BaseHTTPServer and doctest, which fit the model I described above quite nicely.

I think it would be useful to let these use the logging module even without the proposed setdefaultlogger() etc. architecture.

I determined that no changes are necessary for the cgi module, since it doesn't even use the logging functions it defines (initlog, dolog, etc.).

They're intended for use by CGI applications. They should probably be rewritten to direct the calls to the logging module.

I'm currently working on a way to replicate imaplib's command history using LogRecords, which may call for a new type of logging.BufferingHandler subclass that maintains a circular queue of LogRecords.

That might be a useful feature on its own for the logging package. Note though, that at a grander scale, the RotatingFileLogger class already supports a way to keep only the last N messages (for much larger N, though :-).

And the distutils package seems prepped for using the logging package already.

The oddball is unittest. Unittest is what motivated me to bring up the logging issue in the first place, but I can't (or more precisely - am not in a position to) decide what direction that module should take w.r.t. if and how it should use the logging package. Should the TextTestRunner class be modified to use a logging.Logger instead of a WritelnDecorator, or should a separate TestRunner subclass (a LoggingTestRunner perhaps) be defined instead? I'm hoping that Steve Purcell or others may have some thoughts on this.

To reach Steve, you should probably post to the PyUnit list (sorry, you'll have to Google for details, I only vaguely recall that there is such a list).

Personally, I think that unittest is such a special case that it might be best to leave it alone...

I would also appreciate hearing any alternatives to the approach I've suggested, as well as comments on the strengths and weaknesses of each.

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list