[Python-Dev] A new way to configure logging (original) (raw)

Vinay Sajip vinay_sajip at yahoo.co.uk
Wed Oct 7 16:49:44 CEST 2009

At present, configuration of Python's logging package can be done in one of two ways:

  1. Create a ConfigParser-readable configuration file and use logging.config.fileConfig() to read and implement the configuration therein.
  2. Use the logging API to programmatically configure logging using getLogger(), addHandler() etc.

The first of these works for simple cases but does not cover all of the logging API (e.g. Filters). The second of these provides maximal control, but besides requiring users to write the configuration code, it fixes the configuration in Python code and does not facilitate changing it easily at runtime.

In addition, the ConfigParser format appears to engender dislike (sometimes strong dislike) in some quarters. Though it was chosen because it was the only configuration format supported in the stdlib at that time, many people regard it (or perhaps just the particular schema chosen for logging's configuration) as 'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent versions of Python of course support an additional battery-included format which can be used for configuration - namely, JSON. Other options, such as YAML, are also possible ways of configuring systems, Google App Engine-style, and PyYAML has matured nicely.

There has also been talk on the django-dev mailing list about providing better support for using Python logging in Django. When it happens (as of course I hope it does) this has the consequence that many new users who use Django but are relatively inexperienced in Python (e.g. in PHP shops which are moving to Django) will become exposed to Python logging. As Django is configured using a Python module and use of ConfigParser-style files is not a common approach in that ecosystem, users will find either of the two approaches outlined above a particular pain point when configuring logging for their Django applications and websites, unless something is done to avoid it.

All three of the contenders for the title of "commonly found configuration mechanism" - JSON, YAML and Python code - will be expressible, in Python, as Python dicts. So it seems to make sense to add, to logging.config, a new callable bound to "dictConfig" which will take a single dictionary argument and configure logging from that dictionary.

An important facet of implementing such a scheme will be the format or schema which the dictionary has to adhere to. I have started working on what such a schema would look like, and if people here think it's a good idea to go ahead with this, I'll provide the details of the schema on a separate post which I'll also cross-post on comp.lang.python so that I can get feedback from there, too.

In outline, the scheme I have in mind will look like this, in terms of the new public API:

class DictConfigurator: def init(self, config): #config is a dict-like object (duck-typed) import copy self.config = copy.deepcopy(config)

def configure(self):
    #  actually do the configuration here using self.config

dictConfigClass = DictConfigurator

def dictConfig(config): dictConfigClass(config).configure()

This allows easy replacement of DictConfigurator with a suitable subclass where needed.

What's the general feeling here about this proposal? All comments and suggestions will be gratefully received.

Regards,

Vinay Sajip

More information about the Python-Dev mailing list