| msg52973 - (view) |
Author: Chris Leary (cdleary) |
Date: 2007-08-01 08:14 |
| Streams (read: files) are opened in potentially destructive modes when loading logging configurations. This patch provides library users with a method of postponing the opening of streams. The method is simple, powerful, and should be fully compatible with existing infrastructure. The StreamHandler now allows a callback function to be passed as an initialization parameter (@param strm). This function must be callable with no arguments and return a valid stream. A function passed in this manner can be initialized via the initialize_stream() method call; however, if the user chooses not to call initialize_stream() explicitly, the stream is initialized on the first attempt to emit a record. To take advantage of this ability, the FileHandler has been extended with callback-passing functionality. If the delay_fh initialization parameter is asserted, a file-opening closure is passed to the StreamHandler (superclass) initialization method. This ability has also been extended to the module's fileConfig() function. Though fileConfig()'s optional delay_fh parameter, the user may load a configuration from file without undesired file handle opening effects. This is particularly useful when running many instances of the same program on a single file system -- the user now has the ability to change (at runtime) FileHandlers that would otherwise be opened in 'w' mode, clobbering the other instantiation's logs. Another, seemingly tangential portion of the patch adds a kwargs specifier to the configuration file. This is to add the flexibility of adding a delay_fh keyword indicator without specifying other arguments unnecessarily as part of 'args'. It still follows the 1.5 compatible apply() syntax, and it only takes another 4 lines ;) Implementation Notes: - Tested in regtest and with an application I wrote that's quite logging-intensive, and both check out OK so far. - I broke some functions I was working on into helpers for readability, as they were getting too long. They're all initialization functions that shouldn't be performance critical. - Private functions that I added weren't given default values for the delay_fh parameter, since nobody knows about them; however, I gave the private functions that were already there a default value for the delay_fh parameter just to be extra safe and backwards compatible. Hopefully nobody's using those anyway. - I tried my best to follow existing style -- feel free to yell at me if I missed it somewhere :D - Chris Leary |
|
|
| msg55747 - (view) |
Author: Vinay Sajip (vinay.sajip) *  |
Date: 2007-09-08 12:01 |
| Thanks for the patch. I think it would be simpler to just implement an optional delay parameter in FileHandler and its subclasses: then the configuration stuff need not be touched at all, since the delay parameter can be specified in the configuration file. There would then be no need for a callback or a file-opening closure: If delay is False (the default), FileHandler would behave as it does now. If True, then the open code in FileHandler's constructor would be deferred until the time emit() was called on the instance. Would this meet your needs? |
|
|
| msg55772 - (view) |
Author: Chris Leary (cdleary) |
Date: 2007-09-10 01:28 |
| Hi Vinay, I was actually trying to address a use case where the delay_fh option in the fileConfig() would be necessary. Let's say I'm running a simulator that I run many instances of at once. The logging configuration is extensive, so I want to use a configuration file; however, I don't want any existing log files to be clobbered as soon as I run fileConfig() -- I want to run fileConfig() to load the configuration, then remove/modify the handlers /before/ they touch the file handles. If fileConfig has no delay_fh option, fileConfig() would create the FileHandlers with delay_fh as False; therefore, adding the option to the FileHandler alone isn't enough to fix this use case. Let me know if this is unclear, or if I should provide a more concrete example. |
|
|
| msg55776 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2007-09-10 08:54 |
| I'm not sure I agree with "If fileConfig has no delay_fh option, fileConfig() would create the FileHandlers with delay_fh as False". fileConfig will create the FileHandlers using the args= section entry to pass to the constructor. Hence, by specifying True for this parameter, you get the FileHandlers to delay opening the files. In fact you get more control - you can specify True for some FileHandlers and False for others. The relevant code is in config.py in _install_handlers: ... klass = eval(klass, vars(logging)) args = cp.get(sectname, "args") args = eval(args, vars(logging)) h = apply(klass, args) ... |
|
|
| msg55780 - (view) |
Author: Chris Leary (cdleary) |
Date: 2007-09-10 14:26 |
| Ah, I see what the miscommunication is now. Yes, implementing the feature as you suggested would give the user the ability to specify the delayed handle-opening /within/ the configuration file. What I had intended was a way to load/inspect a file configuration without it having any handle-opening side affects, even if it were a totally unknown configuration file. If you want to assume a good trust relationship with the contents of a file configuration that you're loading (as in, you trust that it won't clobber any files that you still want), then that works well. I just assumed it would be useful to open a file configuration in an "untrusted" way (via delay_fh) in order to inspect/modify it. It seems to me to put more power in the hands of the loader of the configuration, rather than leaving the loader subject to the potentially "bad" contents of the configuration file. If you think I'm being too paranoid or feel that this is too obscure a use case, feel free to implement it how you see fit! :) |
|
|
| msg61628 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2008-01-24 12:44 |
| Added optional delay argument to FileHandler and subclasses. |
|
|