[Python-Dev] Python 3.0.1 (io-in-c) (original) (raw)
Antoine Pitrou solipsis at pitrou.net
Tue Jan 27 23:44:49 CET 2009
- Previous message: [Python-Dev] Python 3.0.1
- Next message: [Python-Dev] Python 3.0.1 (io-in-c)
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Raymond Hettinger <python rcn.com> writes:
What is involved in finishing io-in-c?
Off the top of my head:
- fix the _ssl bug which prevents some tests from passing (issue #4967)
- clean up io.py (and decide what to do with the remaining Python code: basically, the parts of StringIO which are implemented in Python)
- of course, test in various situations, review the code, suggest possible improvements...
Now here are some performance figures. Text I/O is done in utf-8 with universal newlines enabled:
=== I/O in C ===
** Binary input **
[ 400KB ] read one unit at a time... 1.64 MB/s [ 400KB ] read 20 units at a time... 27.2 MB/s [ 400KB ] read 4096 units at a time... 845 MB/s
[ 20KB ] read whole contents at once... 924 MB/s [ 400KB ] read whole contents at once... 883 MB/s [ 10MB ] read whole contents at once... 980 MB/s
[ 400KB ] seek forward one unit at a time... 0.528 MB/s [ 400KB ] seek forward 1000 units at a time... 516 MB/s [ 400KB ] alternate read & seek one unit... 1.33 MB/s [ 400KB ] alternate read & seek 1000 units... 490 MB/s
** Text input **
[ 400KB ] read one unit at a time... 2.28 MB/s [ 400KB ] read 20 units at a time... 29.2 MB/s [ 400KB ] read one line at a time... 71.7 MB/s [ 400KB ] read 4096 units at a time... 97.4 MB/s
[ 20KB ] read whole contents at once... 108 MB/s [ 400KB ] read whole contents at once... 112 MB/s [ 10MB ] read whole contents at once... 89.7 MB/s
[ 400KB ] seek forward one unit at a time... 0.0904 MB/s [ 400KB ] seek forward 1000 units at a time... 87.4 MB/s
** Binary append **
[ 20KB ] write one unit at a time... 0.668 MB/s [ 400KB ] write 20 units at a time... 12.2 MB/s [ 400KB ] write 4096 units at a time... 722 MB/s [ 10MB ] write 1e6 units at a time... 1529 MB/s
** Text append **
[ 20KB ] write one unit at a time... 0.983 MB/s [ 400KB ] write 20 units at a time... 16 MB/s [ 400KB ] write 4096 units at a time... 236 MB/s [ 10MB ] write 1e6 units at a time... 261 MB/s
** Binary overwrite **
[ 20KB ] modify one unit at a time... 0.677 MB/s [ 400KB ] modify 20 units at a time... 12.1 MB/s [ 400KB ] modify 4096 units at a time... 382 MB/s
[ 400KB ] alternate write & seek one unit... 0.212 MB/s [ 400KB ] alternate write & seek 1000 units... 173 MB/s [ 400KB ] alternate read & write one unit... 0.827 MB/s [ 400KB ] alternate read & write 1000 units... 276 MB/s
** Text overwrite **
[ 20KB ] modify one unit at a time... 0.296 MB/s [ 400KB ] modify 20 units at a time... 5.69 MB/s [ 400KB ] modify 4096 units at a time... 151 MB/s
=== I/O in Python (branches/py3k) ===
** Binary input **
[ 400KB ] read one unit at a time... 0.174 MB/s [ 400KB ] read 20 units at a time... 3.44 MB/s [ 400KB ] read 4096 units at a time... 246 MB/s
[ 20KB ] read whole contents at once... 443 MB/s [ 400KB ] read whole contents at once... 216 MB/s [ 10MB ] read whole contents at once... 274 MB/s
[ 400KB ] seek forward one unit at a time... 0.188 MB/s [ 400KB ] seek forward 1000 units at a time... 182 MB/s [ 400KB ] alternate read & seek one unit... 0.0821 MB/s [ 400KB ] alternate read & seek 1000 units... 81.2 MB/s
** Text input **
[ 400KB ] read one unit at a time... 0.218 MB/s [ 400KB ] read 20 units at a time... 3.8 MB/s [ 400KB ] read one line at a time... 3.69 MB/s [ 400KB ] read 4096 units at a time... 34.9 MB/s
[ 20KB ] read whole contents at once... 70.5 MB/s [ 400KB ] read whole contents at once... 81 MB/s [ 10MB ] read whole contents at once... 68.7 MB/s
[ 400KB ] seek forward one unit at a time... 0.0709 MB/s [ 400KB ] seek forward 1000 units at a time... 67.3 MB/s
** Binary append **
[ 20KB ] write one unit at a time... 0.15 MB/s [ 400KB ] write 20 units at a time... 2.88 MB/s [ 400KB ] write 4096 units at a time... 346 MB/s [ 10MB ] write 1e6 units at a time... 728 MB/s
** Text append **
[ 20KB ] write one unit at a time... 0.0814 MB/s [ 400KB ] write 20 units at a time... 1.51 MB/s [ 400KB ] write 4096 units at a time... 118 MB/s [ 10MB ] write 1e6 units at a time... 218 MB/s
** Binary overwrite **
[ 20KB ] modify one unit at a time... 0.123 MB/s [ 400KB ] modify 20 units at a time... 2.34 MB/s [ 400KB ] modify 4096 units at a time... 213 MB/s
[ 400KB ] alternate write & seek one unit... 0.0816 MB/s [ 400KB ] alternate write & seek 1000 units... 71.4 MB/s [ 400KB ] alternate read & write one unit... 0.0448 MB/s [ 400KB ] alternate read & write 1000 units... 41.1 MB/s
** Text overwrite **
[ 20KB ] modify one unit at a time... 0.0723 MB/s [ 400KB ] modify 20 units at a time... 1.36 MB/s [ 400KB ] modify 4096 units at a time... 88.3 MB/s
Regards
Antoine.
- Previous message: [Python-Dev] Python 3.0.1
- Next message: [Python-Dev] Python 3.0.1 (io-in-c)
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]