[Python-Dev] Consistent logging in the standard library (original) (raw)

Matthew F. Barnes mfb at lotusland.dyndns.org
Sun Sep 7 23:13:02 EDT 2003

Having just read the "Use of the logging package in the standard library" section of Brett's latest python-dev summary [1], I thought I might try to generate some discussion on this topic since I brought it up in the first place.

There was actually a bit of a longer discussion of this topic on comp.lang.python [2], where A.M. Kuchling pointed out a number of good issues that will need to be considered and suggested that I write a PEP on the topic. In fact I've been working on one since, but it's not yet ready to be submitted (it's my first, so it's taking awhile).

The basic idea in the PEP is this: maintain a level of indirection between the logger objects and the modules or class instances invoking them, so that custom application-specific loggers can be substituted for the "standard" ones.

The method for doing so is modelled after the socket module's new timeout functionality [3], where each module that performs logging defines a pair of module-level functions named getdefaultlogger() and setdefaultlogger() that return and accept (respectively) a logging.Logger instance. The initial default logger for each of these modules is named after the module itself (i.e. logging.getLogger(name)) and its behavior mimics that of the Python 2.3 version of the module in terms of format and destination of the logging messages.

So an application that wants to override the logging behavior of a standard library module can do so simply by passing a customized logger object to the module's setdefaultlogger() function. This works out even better for modules where all the logging is encapsulated in a class.

The PEP calls for each class that performs logging to define an attribute called self.logger, and upon instantiation initialize self.logger to the module's current default logger (again, modelled after how the timeout value for a socket object is initialized). This allows each instance of a class to have its own unique logging behavior.

The first item for discussion is whether this seems like a reasonable approach. The second is identifying which standard library modules are candidates for modification using this approach. I've identified the following by simply searching for the word "logging" in the source code, but I'm not sure whether this list is complete:

    asyncore
    BaseHTTPServer
    cgi
    doctest
    imaplib
    unittest
    the distutils package

I have patches ready for asyncore and BaseHTTPServer and doctest, which fit the model I described above quite nicely. I determined that no changes are necessary for the cgi module, since it doesn't even use the logging functions it defines (initlog, dolog, etc.). I'm currently working on a way to replicate imaplib's command history using LogRecords, which may call for a new type of logging.BufferingHandler subclass that maintains a circular queue of LogRecords. And the distutils package seems prepped for using the logging package already.

The oddball is unittest. Unittest is what motivated me to bring up the logging issue in the first place, but I can't (or more precisely - am not in a position to) decide what direction that module should take w.r.t. if and how it should use the logging package. Should the TextTestRunner class be modified to use a logging.Logger instead of a _WritelnDecorator, or should a separate TestRunner subclass (a LoggingTestRunner perhaps) be defined instead? I'm hoping that Steve Purcell or others may have some thoughts on this.

I would also appreciate hearing any alternatives to the approach I've suggested, as well as comments on the strengths and weaknesses of each.

Matthew Barnes

[1]

<http://mail.python.org/pipermail/python-dev/2003-September/037938.html> [2] <http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF-8&threadm=3a8e83d2.0308182217.7ccaf883%40posting.google.com&rnum=1&prev=/groups%3Fq%3DUnification%2BPython%2BBarnes%26hl%3Den%26lr%3Dlang_en%26ie%3DUTF-8%26oe%3DUTF-8%26selm%3D3a8e83d2.0308182217.7ccaf883%2540posting.google.com%26rnum%3D1> [3] <http://www.python.org/doc/2.3/lib/module-socket.html>

More information about the Python-Dev mailing list