[Python-Dev] Updated PEP 454 (tracemalloc): no more metrics! (original) (raw)

Charles-François Natali cf.natali at gmail.com
Thu Oct 24 19:09:53 CEST 2013

2013/10/24 Kristján Valur Jónsson <kristjan at ccpgames.com>:

Now, I would personally not truncate the stack, because I can afford the memory, but even if I would, for example, to hide a bunch of detail, I would want to throw away the lower detals of the stack. It is unimportant to me to know if memory was allocated in ...;itertools.py;logging.py;stringutil.py but more important to know that it was allocated in main.py;databaseengine.py;enginesettings.py;...

Well, maybe to you, but if you look at valgrind for example, it keeps the top of the stack: and it makes a lot of sense to me, since otherwise you won't be able to find where the leak occurred.

Anyway, since the stack depth is a tunable parameter, this shouldn't be an issue in practice: just save the whole stack.

2013/10/24 MRAB <python at mrabarnett.plus.com>:

When I was looking for memory leaks in the regex module I simply wrote all of the allocations, reallocations and deallocations to a log file and then parsed it afterwards using a Python script. Simple, but effective.

We've all done that ;-)

1) really, all that is required in terms of data is the traceback.gettraces() function. Further, it need not return addresses since they are not required for analysis. It is sufficient for it to return a list of (traceback, size, count) tuples.

Sure. Since the beginning, I'm also leaning towards a minimal API, and let third-party tools do the analysis.

It makes a lot of sense, since some people will want just basic snapshot information, some others will want to compute various statistics, some others will want to display the result in a GUI...

But OTOT, it would be too bad not to ship the stdlib with a basic tool to process data, to as to make it usable out-of-the box.

And in this regard, we should probably mimick what's done for CPU profiling: there are both low-level profiling data gathering infrastructure (profile and cProfile), but there's also a pstats.Stats class allowing basic operations/display on this raw data.

That's IMO a reasonable balance.

cf

More information about the Python-Dev mailing list