[Python-Dev] Unpickling memory usage problem, and a proposed solution (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Fri Apr 23 23:56:19 CEST 2010

Dan Gindikin <dgindikin gmail.com> writes:

Antoine Pitrou <solipsis pitrou.net> writes: > Does cPickle bytecode have some kind of NOP instruction? > You could keep track of which PUTs weren't necessary and zero them out at the > end. It would be much cheaper than writing a whole other "optimized" stream. For a large file, I'm not sure it is much faster to edit it in place than to rewrite it, and also since it's a large file, you are going to probably have it compressed, in which case you are out of luck anyway.

Depends whether you really care about disk occupation or not. Disk space is often cheap (much more so than RAM). Also, I'm quite sure overwriting a couple of blocks in a file is cheaper than rewriting it entirely. Of course, if you must overwrite every other block, it might not be true anymore :-)

Regards

Antoine.

More information about the Python-Dev mailing list