[Python-Dev] Re: RELEASED Python 2.3.1 (original) (raw)

Barry Warsaw barry at python.org
Sun Sep 28 01:24:27 EDT 2003

On Sat, 2003-09-27 at 13:48, "Martin v. Löwis" wrote:

5. There is a power outage. This is the case where fsync should help: everything up to the write barrier is on disk. Of course, if the disk drive itself has write caching, fsync may have completed without the data being on the disk (this would be an fsync bug, but I believe Linux suffers from this particular bug).

So in short, fsync(2) helps only in case of a power outage; for normal operation, it is not needed. In the case of a power outage, it is doubtful whether it has the desired effect.

This was the situation that folks reported causing message loss in Mailman 2.1.2. Those folks also reported that adding the fsync helped them (I guess a UPS would have helped too ). I've added it to MM2.1.3, but I haven't enabled it by default. I'm still not sure the specific problem can be fixed in better ways (e.g. running on a more reliable host, running it on a sync'ing filesystem, or maybe opening the file with O_SYNC). Also, I did some benchmarking with fsync's added and on the system I tested I saw a 97% performance hit when always fsync'ing. That made me nervous.

Before I enable it unconditionally, I'd want to be more certain that it's necessary, or useful for the majority of sites, and that the performance hit is either worth it, not as bad as my benchmarks showed, or at least not as observably bad under real-world conditions.

-Barry

More information about the Python-Dev mailing list