Issue 979252: Trap OSError when calling RotatingFileHandler.doRollover (original) (raw)

I use the RotatingFileHandler in most of my scripts. The script will crash if the RotatingFileHandler encounters a locked log file.

I would like to see something like:

def emit(self, record):
    """
    Emit a record.

    Output the record to the file, catering for rollover 

as described in init(). """ if self.maxBytes > 0: # are we rolling over? msg = "%s\n" % self.format(record) self.stream.seek(0, 2) #due to non-posix- compliant Windows feature if self.stream.tell() + len(msg) >= self.maxBytes: try: self.doRollover() except Exception: logging.FileHandler.emit(self, 'Failed to doRollover.') logging.FileHandler.emit(self, record)

My version of Python (2.3.2) had the wrong docstring as well, referring to a non-existent setRollover.

Logged In: YES user_id=975020

Richard, I have also had the "joy" of encountering this bug (on a windows 2000 machine), although I am still baffled at what could be locking the file.

Our testing did not reveal this bug and now we have some code in a production environment that is causing me grief, so I had to come up with a solution quickly and I though that I might share it with you (and others).

I don't believe that your solution will solve the problem. In doRollover the stream is closed and then the files are manipulated. Since this is where the errors will occur simply calling the emit method again will only cause more exceptions to be generated. Also, since the stream is never opened again, you would never get any log messages until you restarted. Sorry, after looking a bit more it seems like you will get tracebacks printed to the standard error (where ever you have designated that to be).

What I am doing is replacing my version of doRollover with this version; def doRollover(self): """ Do a rollover, as described in init(). """

    self.stream.close()
    openMode = "w"
    if self.backupCount > 0:
        try:
            for i in range(self.backupCount - 1, 0, -1):
                sfn = "%s.%d" % (self.baseFilename, i)
                dfn = "%s.%d" % (self.baseFilename, i + 1)
                if os.path.exists(sfn):
                    #print "%s -> %s" % (sfn, dfn)
                    if os.path.exists(dfn):
                        os.remove(dfn)
                    os.rename(sfn, dfn)
            dfn = self.baseFilename + ".1"
            if os.path.exists(dfn):
                os.remove(dfn)
            os.rename(self.baseFilename, dfn)
            #print "%s -> %s" % (self.baseFilename, dfn)
        except:
            ## There was some problem with the file

manipulationg above. ## If someone has some time, maybe they can figure out what errors ## can occur and the best way to deal with all of them. ## For my purposes, I'm just going to try and re-open ## the original file. openMode = "a" try: self.stream = open(self.baseFilename, openMode) except: ## There was some problem opening the log file. ## Again, someone with more time can look into all of the possible ## errors and figure out how best to handle them. ## I'm just going to try and open a 'problem' log file. ## The idea is that this file will exist long enough for the problems ## above to go away and the next time that we attempt to rollover ## it will work as we expect. gotError = True count = 1 while gotError: try: problemFilename = self.baseFilename + ".pblm." + str(count) self.stream = open(problemFilename, openMode) gotError = False except: count += 1

If my logic works out, this should always open a file for logging so the logging should never crash and no log messages will be lost. Hopefully you will find this helpful.

Logged In: YES user_id=308438

Easy things first - the docstring has been corrected in a later release.

I believe that when an exception occurs during logging, one of two things should happen: if raiseExceptions is 0, then the exception should be swallowed silently (typically, for production environments). This is because logging-related errors should not be treated in the same way as application errors (it's a pseudo-philosophical point, and not everyone will agree). If raiseExceptions is 1 (development environments), the exception should be raised.

I think the solution given by David London (groodude) may be fine for specific applications, but not sufficiently general to be adopted as is. I will check in changes to CVS which will call handleError() when an exception occurs during emit(). It's best to derive from RotatingFileHandler and put any specific handling you want into an overridden handleError().