[Python-Dev] Using logging in the stdlib and its unit tests (original) (raw)

Glenn Linderman v+python at g.nevcal.com
Thu Dec 9 10:29:51 CET 2010

On 12/9/2010 12:26 AM, Vinay Sajip wrote:

Glenn Linderman<v+python g.nevcal.com> writes:

> Or what am I missing? That threads are not necessarily dedicated to apps, in a real world setting. Depending on the server implementation, a single thread could be asked to handle requests for different apps over its lifetime. So the only way of knowing which threads are currently servicing a particular app is to maintain a set of them.

Agreed, they are not necessarily dedicated to apps. But while they are running the app, they have the appname in their thread local storage, no? So if a thread has the appname in its thread local storage, is it not servicing that app, at that moment? And if it is, why is that insufficient? That is my question, and you've sidestepped it.

And one more thing: the filters forboth apps are called for a given request. One will return True, the other will return False.

Bear in mind that the intention of the post is to be didactic, so it's possible there are some improvements that could be made when implementing for production.

OK, I just learned what "didactic" meant, so you've taught me something.

I understood that both filters would be called. I understood, that in production, it would probably not be necessary to log messages to both the app log and the global log, that the global log would be there just for things that are not app-specific.

But I still don't see a need to maintain a set of threads running the app in the app object, if the app keeps track of what app it is running, in a spot that is accessible to the filter (which it seems to be). I don't see how it is beneficial to keep track of two separate data structures that could serve the same purpose.

I realize you designed this in 10 minutes, and probably took twice that long to write it up, so didn't necessarily analyze it for efficiency.
But I'm asking for that analysis now, if there is any real need for the app to track the set of threads, for the purpose of the problem being solved? I understand there might be other reasons for which that might be useful, but for the logging it simply seems to be inefficient redundancy... and if it isn't, then I don't understand the example, yet, and I'm trying to.

So since you hand-waved, instead of giving a straight answer, and then maybe your second message was an attempt to back-pedal a bit, not sure, I went ahead and downloaded your code, made changes to remove the set of threads as outlined (see modified functions below), and it seems to run just as correctly. I thought it was an obvious question, while trying to understand the code, and maybe learn about the logger, and I guess I have, a little. And maybe some other things too. Please further explain if I am still missing something. Under what conditions of the problem you were solving does the code below fail?

class InjectingFilter(logging.Filter): def init(self, app): self.app = app

 def filter(self, record):
     record.method = tlocal.request.method
     record.ip = tlocal.request.ip
     record.appName = tlocal.appName
     return record.appName == self.app.name
     # tname = threading.currentThread().getName()
     # return tname in self.app.threads

class WebApp: def init(self, name): self.name = name self.threads = set() handler = logging.FileHandler(name + '.log', 'w') f = InjectingFilter(self) handler.setFormatter(formatter) handler.addFilter(f) root.addHandler(handler) self.num_requests = 0

 def process_request(self, request):
     tlocal.request = request
     tlocal.appName = self.name
     # tname = threading.currentThread().getName()
     # self.threads.add(tname)
     self.num_requests += 1
     try:
         logger.debug('Request processing started')
         webapplib.useful()
         logger.debug('Request processing finished')
     finally:
         pass
         # self.threads.remove(tname)

-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20101209/0097cd7e/attachment-0001.html>

More information about the Python-Dev mailing list