Message 111399 - Python tracker (original) (raw)

Won't happen for Python 2.x but the idea is good. The original patch won't apply for Py3k so I implemented this approach from scratch for Python 3.2. Patch included.

Some microbenchmarks of mine show:

• a 3-5% slowdown for very small files (we can ignore this since it's still a blink of an eye)
• a 10% speedup in usual cases (files the size of a typical Pylons app configuration or a typical buildout)
• more so in unrealistic ones (many values with > 100 lines each)

Brett, a quick summary of changes:

• the idea to introduce a list of lines instead of gluing '%s\n%s' during file reading was introduced
• this approach needs an additional phase of joining the lists to strings after reading is done.
• I used itertools.chain to add the default section for iteration purposes without having to create a separate list. I hope the additional dependency on itertools is no problem.
• I reformatted a bit the changes by fred.drake in r78233 (he added support for valueless keys)
• some other slight formatting changes apply as well
• in the tests I've added a testcase for a file that uses multi-line values heavily. The file is generated temporarily since it's 127kB in size. Creation takes a fraction of a second so shouldn't be noticeable.