[Python-Dev] CHUNKSIZE (original) (raw)

Martin v. Loewis martin@v.loewis.de
28 Apr 2002 01:19:52 +0200

Michael Gilfix <mgilfix@eecs.tufts.edu> writes:

This comment is more of a question.. Does the chunksize not have to do with how the filesystem stores blocks?

Yes, but probably in non-obvious ways. Many file systems attempt to allocate subsequent blocks in a file into subsequent sectors of the disk. As a result, you can get lots of data in a single rotation of the disk. If you are going to use all the data, it is better to transfer them all in one chunk - the kernel needs to setup IO only for a single operation, which then completes at the burst speed of the device. If the disk repositions heads between reads, performance goes down.

Regards, Martin