[Python-Dev] cProfile and threads (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Tue Aug 17 12:03:49 CEST 2010


2010/8/17 Kristján Valur Jónsson <kristjan at ccpgames.com>:

Hello there.

I‘d like to draw your attention to two feature requests / patches that I‘ve subbmitted: http://bugs.python.org/issue9609 http://bugs.python.org/issue9622 These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python.

Both look like good ideas to me (multi-threaded profiling and debugging is fairly painful and it would be good to be able to do something to improve that situation).

The first patch makes lsprof, the engine behind cProfile, multi-stack aware.  This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered.

I'm curious as to the memory impact this has on the profiler (it obviously can't be too bad if you're able to run it against your live servers).

The second patch allows to set the trace/profile function in python globally, so that all threads are affected.  This is essential if you want to take a profililng snapshot of a running application.

One minor quibble here is that I would suggest using "global=False" in your docstring signatures.

Both patches seem to be missing updates to the relevant documentation. I expect this would be difficult to unit test properly, but at least some basic tests to check that the new global configuration of settrace and setprofile don't blow, and that a profiler can be shared between two threads would be good.

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia



More information about the Python-Dev mailing list