[Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() (original) (raw)

Bob Ippolito bob at redivi.com
Thu Feb 26 22:21:53 EST 2004

On Feb 26, 2004, at 10:03 PM, Raymond Hettinger wrote:

I had been led astray because I was experimenting with using cStringIO.writelines() as a basis for implementing str.join() for general iterables without creating an intermediate tuple. Right now, ''.join(it) will unexpectedly consume much more memory than really needed. [Jeremy Fincher] This sounds like a cool idea, are you still going to implement it? One way or another, I'll make str.join() smarter and faster than it is now. Which way proves to be better is still open.

In trying to time this in pure python, I discovered that cStringIO.writelines doesn't take generators.. apparently only objects that support len(...) are allowed

Currently, it seems that ''.join is faster than cStringIO.writelines anyway, even if you just pass it a big list.

from cStringIO import StringIO def join2(sep, seq): sio = StringIO() if not sep: sio.writelines(seq) else: # does not work def join2(): yield seq.next() for s in seq: yield sep yield s sio.writelines(join2()) return sio.getvalue()

import time from itertools import repeat import operator def timeit(fn, *args, **kwargs): lst = [] for ig in xrange(100): t0 = time.time() fn(*args, **kwargs) lst.append(time.time() - t0) print fn.name print '', 'max:', max(lst) print '', 'min:', min(lst) print '', 'avg:', reduce(operator.add, lst)/len(lst)

if name == 'main': strings = [' ' * 1000] * 1000 print 'no separator' print '------------' timeit(join2, '', strings) timeit(''.join, strings)

[crack:~] bob% python strio.py no separator

join2 max: 0.0453701019287 min: 0.0157110691071 avg: 0.0181557154655 join max: 0.0425598621368 min: 0.00337600708008 avg: 0.00432039260864

More information about the Python-Dev mailing list