[Python-Dev] Evil reference cycles caused Exception.traceback (original) (raw)

Martin (gzlist) gzlist at googlemail.com
Mon Sep 18 14:55:02 EDT 2017

Thanks for working on this and writing up the details Victor.

For those confused about why this matters, routinely having every object in your application participating in one (or more) giant reference cycles makes reasoning about and fixing resource issues very difficult.

For instance, a while back I made some changes in Bazaar so that each TestCase instance was dereferenced after being run:

<https://bugs.launchpad.net/bzr/+bug/613247>

Which mattered when running ~25000 tests, to keep peak memory usage sane. Also there were various other object lifetime issues, and tracking down which specific test failed to join a thread, or close a file on Windows was much simpler after making sure cleanup actually happened at the time a test was deleted.

Keeping more stuff alive for longer periods also makes peak memory requirements higher, and the gc has to do more work each time.

On 18/09/2017, Victor Stinner <victor.stinner at gmail.com> wrote:

Ideally, CPython 3.x should never create reference cycles. Removing Exception.traceback is the obvious "fix" for the issue. But I expect that slowly, a lot of code started to rely on the attribute, maybe even for good reasons :-) A more practical solution would be to log a warning. Maybe the garbage collector can emit a warning if it detects an exception part of a reference cycle? Or maybe detect frames?

Logging a warning is unlikely to be practical. I had an optional test flag that complained about reference cycles, and it produced a lot of output.

Martin

More information about the Python-Dev mailing list