[Python-Dev] smarter temporary file object (SF #415692) (original) (raw)

dustin at v.igoro.us dustin at v.igoro.us
Wed Jan 3 06:27:40 CET 2007

I came across a complaint that PEP 0042 had become a graveyard of neglected ideas, and decided to have a look through and implement something. Creating a smarter temporary file object seemed simple enough.

Oddly, even after GvR re-opened it, I can't post an attachment to that tracker item (it's under "Feature Requests" -- does it need to get moved to "Patches" first?), but the implementation is short, so it's included below. This is intended to be appended to Lib/tempfile.py (and thus assumes that module's globals are present).

I would appreciate it if the gurus of python-dev could take a peek and let me know if this is unsuitable or incorrect for any reason. It's not the most straightforward implementatio -- I used the optimization techniques I found in TemporaryFile.

If this looks good, I'll prepare a patch against trunk, including an additional chunk of documentation and a unit test.

Dustin

                 -----cut-here-----

try: from cStringIO import StringIO except: from StringIO import StringIO

class SpooledTemporaryFile: """Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed. """ _rolled = False

def __init__(self, max_size=0, mode='w+b', bufsize=-1,
             suffix="", prefix=template, dir=None):
    self._file = StringIO()
    self._max_size = max_size
    self._TemporaryFileArgs = (mode, bufsize, suffix, prefix, dir)

def _check(self, file):
    if self._rolled: return
    if file.tell() > self.__dict__['_max_size']:
        self._rollover(file)

def _rollover(self, file):
    args = self.__dict__['_TemporaryFileArgs']
    self.__dict__.clear() # clear attributes cached by __getattr__
    newfile = self._file = TemporaryFile(*args)
    newfile.write(file.getvalue())
    newfile.seek(file.tell(), 0)
    self._rolled = True
    
    # replace patched functions with the new file's methods
    self.write = newfile.write
    self.writelines = newfile.writelines
    self.fileno = newfile.fileno

def write(self, s):
    file = self.__dict__['_file']
    rv = file.write(s)
    self._check(file)
    return rv

def writelines(self, iterable):
    file = self.__dict__['_file']
    rv = file.writelines(iterable)
    self._check(file)
    return rv

def fileno(self):
    self._rollover(self.__dict__['_file'])
    return self.fileno()

def __getattr__(self, name):
    file = self.__dict__['_file']
    a = getattr(file, name)
    if type(a) != type(0):
        setattr(self, name, a)
    return a

More information about the Python-Dev mailing list