[Python-Dev] Hotshot (original) (raw)

Walter Dörwald walter at livinglogic.de
Wed Jan 28 08:55:38 EST 2004

Fred L. Drake, Jr. wrote:

Skip Montanaro writes: > It seems to me it might be simpler to just write the profile file through > the gzip module and teach the hotshot.stats.load() function to recognize > such files and uncompress accordingly.

The problem with this is that the low-level log reader and writer are in C rather than Python (using fopen(), fwrite(), etc.). We could probably work out a reader that gets input buffers from an arbitrary Python file object, and maybe we could handle writing that way, but that does change the cost of each write. HotShot tries to compensate for that, but it's unclear how successful that is.

OK, I've checked both the old profile.py and the new hotshotmain.py (Thanks for checking it in, Skip) with one of my scripts. The script runs for: real 0m6.192s user 0m6.010s sys 0m0.160s when running standalone.

Using the old profile.py I get the following: real 0m46.892s user 0m43.430s sys 0m0.970s

Running with the new hotshotmain.py gives the following run times: real 1m6.873s user 1m5.220s sys 0m0.840s

The size of hotshop.prof is 5104590 bytes. After gzipping it with "gzip -9" the size of hotshop.prof drops to 562467 bytes. So gzipping might help, but dropping filesize from 1 GB to 100 MB still doesn't sound so convincing. And I wonder what would happen to run time.

Bye, Walter Dörwald

More information about the Python-Dev mailing list