Issue 231540: threads and profiler don't work together (original) (raw)

When a new thread is created, it doesn't inherit from the parent thread the trace and profile functions (sys_tracefunc and sys_profilefunc in PyThreadState), so multithreaded programs can't easily be profiled.

This may be by design for safety/complexity sake, but the profiler module should still find some way to function correctly. A temporary (and performance-killing) workaround is to modify the standard profiler to hook into threads to start a new profiler for each new thread, and then merge the stats from a child thread into the parent's when the child thread ends.

Here is sample code that exhibits the problem. Stats are printed only for the main thread because the child thread has no profiling function and therefore collects no stats:

import threading, profile, time

def yo(): for j in range(5): print j,

def go(): threading.Thread(target=yo).start() time.sleep(1)

profile.run('go()')

I don't think this problem still exists now. In the current implementation, there is no "sys_tracefunc" and "sys_profilefunc" in PyThreadState, but "c_profilefunc", "c_profileobj", "c_tracefunc", "c_traceobj" instead. When creating a new thread, the "c_profilefunc" and "c_tracefunc" are inherited from main thread, and the profile function is thread specific, it only collect profile statistic of the executing functions in its own thread, that is, each thread can profile its own executing.

I'd change the example as follows:

def child(): def yo(): for j in range(5): print(j) profile.runctx('yo()', globals(), locals())

def go(): threading.Thread(target=child).start() time.sleep(1)

profile.runctx('go()', globals(), locals())

This will output two profile statistics, one is for main thread, another is for child thread: child(). So if you want to profile a child thread, just call profile.run() in child thread.

So I don't think this is a problem.