[Python-Dev] Unifying trace and profile (original) (raw)

Nicholas Bastin nick.bastin at gmail.com
Thu Feb 23 07:14:12 CET 2006

On 2/21/06, Robert Brewer <fumanchu at amor.org> wrote:

1. Allow trace hooks to receive ccall, creturn, and cexception events (like profile does).

I can easily make this modification. You can also register the same bound method for trace and profile, which sortof eliminates this problem.

2. Allow profile hooks to receive line events (like trace does).

You really don't want this in the general case. Line events make profiling really slow, and they're not that accurate (although many thanks to Armin last year for helping me make them much more accurate). I guess what you require is to be able to selectively turn on events, thus eliminating the notion of 'trace' or 'profile' entirely, but I don't have a good idea of how to implement that at least as efficiently as the current system at the moment - I'm sure it could be done, I just haven't put any thought into it.

3. Expose new sys.gettrace() and getprofile() methods, so trace and profile functions that want to play nice can call sys.settrace/setprofile(None) only if they are the current hook.

Not a bad idea, although are you really running into this problem a lot?

4. Make "the same move" that sys.exitfunc -> atexit made (from a single function to multiple functions via registration), so multiple tracers/profilers can play nice together.

It seems very unlikely that you'll want to have a trace hook and profile hook installed at the same time, given the extreme unreliability this will introduce into the profiler.

5. Allow the core to filter on the "event" arg before hook(frame, event, arg) is called.

What do you mean by this, exactly? How would you use this feature?

6. Unify tracing and profiling, which would remove a lot of redundant code in ceval and sysmodule and free up some space in the PyThreadState struct to boot.

The more events you throw in profiling makes it slow, however. Line events, while a nice thing to have, theoretically, would probably make a profiler useless. If you want to create line-by-line timing data, we're going to have to look for a more efficient way (like sampling).

7. As if the above isn't enough of a dream, it would be nice to have a bytecode tracer, which didn't bother with the flineno logic in maybecalllinetrace, but just called the hook on every instruction.

I'm working on one, but given how much time I've had to work on my profiler in the last year, I'm not even going to guess when I'll get a real shot at looking at that.

My long-term goal is to eliminate profiling and tracing from the core interpreter entirely and implement the functionality in such a way that they don't cost you when not in use (i.e., implement profilers and debuggers which poke into the process from the outside, rather than be supported natively through events). This isn't impossible, but it's difficult because of the large variety of platforms. I have access to most of them, but again, my time is hugely constrained right now for python development work.

-- Nick

More information about the Python-Dev mailing list