[Python-Dev] PEP 454 (tracemalloc) disable ==> clear? (original) (raw)

Victor Stinner victor.stinner at gmail.com
Wed Oct 30 21:40:39 CET 2013

Le 30 oct. 2013 20:58, "Jim Jewett" <jimjjewett at gmail.com> a écrit :

hough if you use a dict internally, that might not be the case.

Tracemalloc uses a {address: trace} duct internally.

If you return it as a list instead of a dict, but that list is NOT in time-order, that is worth documenting

Ok i will document it.

Also, am I misreading the documentation of gettraces() function?

Get traces of memory blocks allocated by Python. Return a list of (size: int, traceback: tuple) tuples. traceback is a tuple of (filename: str, lineno: int) tuples.

So it now sounds like you don't bother to emit de-allocation events because you just remove the allocation from your internal data structure.

I don't understand your question. Tracemalloc does not store events but traces. When a memory block is deallocated, it us removed from the internal dict (and so from get_traces() list).

I still don't see anything here(*) that requires even saving the address, let alone preventing re-use.

The address must be stored internally to maintain the internal dict. See the C code.

(1) Whoa -- memory hog! How can I fix this?

(2) I know -- track allocallocations, with a traceback showing why they were made. (At a minimum, I would like to be able to subclass your tool to do this -- preferably without also keeping the full history in memory.)

What do you mean by "full history" and "subclass your tool"?

(3) Oh, maybe I should skip the ones that really are temporary and get cleaned up. (You make this easy by handling the de-allocs, though I'm not sure those events get exposed to anyone working at the python level, as opposed to modifying and re-compiling.)

If your temporary objects are destroyed before you call get_traces(), you will not see them in get_traces(). I don't understand.

(4) hmm... still too big ... I should use filters. (But will changing those filters while tracing is enabled mess up your current implementation?)

If you call add_filter(), new traces() will be filtered. Not the old ones, as explained in the doc. What do you mean by "mess up"?

(5) Argh. What I really want is to know what gets allocated at times like XXX. I can do that if times-like-XXX only ever occur once per process. I might be able to do it with filters. But I would rather do it by saying "trace on" and "trace off". Maybe even with a context manager around the suspicious places.

I don't understand "times like XXX", what is it?

To see what happened between two lines of code, you can compare two snapshots. No need to disable tracing.

(6) Then, at the end of the run, I would say "give me the info about how much was allocated when tracing was on." Some of that might be going away again when tracing is off, but at least I know what is making the allocations in the first place. And I know that they're sticking around "long enough".

I think you musunderstood how tracemalloc works. You should compile it and play with it. In my opinion, you already have everything in tracemalloc for you scenario.

Under your current proposal, step (5) turns into

set filters trace on ... gettraces serialize to some other storage trace off

s1=take_snapshot() ... s2=take_snapshot() ... diff=s2.statistics("lines", compare_to=s1)

why even have gettraces, as opposed to just takesnapshot? Is there some difference between them, except that a snapshot has some convenience methods and some simple metadata?

See the doc: Snapshot.traces is the result of get_traces().

get_traces() is here is you want to write your own tool without Snapshot.

Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20131030/ce963770/attachment.html>

More information about the Python-Dev mailing list