Issue 33418: Memory leaks in functions (original) (raw)

Issue33418

Created on 2018-05-03 17:54 by jdemeyer, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8058 merged methane,2018-07-03 10:16
PR 15826 merged vstinner,2019-09-10 07:41
PR 16502 merged nascheme,2019-10-01 07:37
Messages (12)
msg316125 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2018-05-03 17:54
This is a memory leak: >>> while True: ... def f(): pass ... f.__doc__ = f This also: >>> while True: ... f = [].append ... f.__module__ = f This is because the classes "function" and "builtin_function_or_method" do not implement tp_clear.
msg320953 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-07-03 10:19
I'm not sure this should be backported to 3.6 and 2.7. While this is a obvious bug, f.__module__ = f seems very unnatural.
msg320954 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2018-07-03 10:23
> While this is a obvious bug, f.__module__ = f seems very unnatural. Sure.
msg321009 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-07-04 02:16
New changeset 3c452404ae178b742967589a0bb4a5ec768d76e0 by INADA Naoki in branch 'master': bpo-33418: Add tp_clear for function object (GH-8058) https://github.com/python/cpython/commit/3c452404ae178b742967589a0bb4a5ec768d76e0
msg350993 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 12:44
This change introduced the possibility to have function objects in an inconsistent state. For example, when calling tp_clear on the function some code must be invoked that tries to ca the function but some fields are NULL causing a crash. I think we should revert this and think better how to protect against this situation.
msg350994 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 12:45
See also https://bugs.python.org/issue38006
msg351013 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-02 15:19
I am going to re-close this until we understand exactly how this is interacting with https://bugs.python.org/issue38006 as this seems more complicated than our first hypothesis.
msg351587 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-10 07:42
"f.__doc__ = f" or "f.__module__ = f" don't make any sense. It looks like a bug triggered on purpose. While it's bad, it's not a big deal. The fix (commit 3c452404ae178b742967589a0bb4a5ec768d76e0) is causing way worse issues than the "artificial" memory leak (reference cycle). I wrote PR 15826 to revert the change in Python 3.8. bpo-38006 root issues are not well understood nor fixed yet.
msg351604 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2019-09-10 09:52
I'm OK to revert it in 3.8. But I am worrying about func.func_closure. Can it create cyclic reference in real life applications?
msg351605 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2019-09-10 09:57
New changeset ccaea525885e41c5f1e566bb68698847faaa82ca by T. Wouters (Victor Stinner) in branch '3.8': Revert "bpo-33418: Add tp_clear for function object (GH-8058)" (GH-15826) https://github.com/python/cpython/commit/ccaea525885e41c5f1e566bb68698847faaa82ca
msg351687 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-10 14:50
> But I am worrying about func.func_closure. Can it create cyclic reference in real life applications? remove.__closure__ is part of a reference cycle in bpo-38006. I like func_clear() (which is still implemented in the master branch), but we need to understand why/how it was possible *crash Python* in bpo-38006.
msg351737 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-09-10 18:58
> It looks like a bug triggered on purpose. Absolutely. It's one of the many small issues that I found while working on PEP 590 and related things.
History
Date User Action Args
2022-04-11 14:59:00 admin set github: 77599
2019-10-01 07:37:55 nascheme set pull_requests: + <pull%5Frequest16095>
2019-09-10 18:58:40 jdemeyer set messages: +
2019-09-10 14:50:52 vstinner set messages: +
2019-09-10 09:57:34 twouters set nosy: + twoutersmessages: +
2019-09-10 09:52:23 methane set messages: +
2019-09-10 07:42:59 vstinner set nosy: + vstinnermessages: +
2019-09-10 07:41:38 vstinner set pull_requests: + <pull%5Frequest15474>
2019-09-02 15:19:27 pablogsal set status: open -> closedresolution: fixedmessages: +
2019-09-02 13:08:30 pablogsal set resolution: fixed -> (no value)
2019-09-02 12:45:38 pablogsal set nosy:methane, serhiy.storchaka, jdemeyer, pablogsalresolution: fixedmessages: +
2019-09-02 12:44:28 pablogsal set status: closed -> openresolution: fixed -> (no value)
2019-09-02 12:44:15 pablogsal set nosy: + pablogsalmessages: +
2018-07-04 02:16:14 methane set status: open -> closedstage: patch review -> resolvedresolution: fixedversions: - Python 2.7, Python 3.6, Python 3.7
2018-07-04 02:16:00 methane set messages: +
2018-07-03 10:23:35 jdemeyer set messages: +
2018-07-03 10:19:55 methane set nosy: + methanemessages: +
2018-07-03 10:16:42 methane set keywords: + patchstage: patch reviewpull_requests: + <pull%5Frequest7667>
2018-05-03 18:33:40 serhiy.storchaka set nosy: + serhiy.storchakaversions: + Python 2.7, Python 3.6, Python 3.7, Python 3.8
2018-05-03 17:54:46 jdemeyer set type: resource usagecomponents: + Interpreter Core
2018-05-03 17:54:23 jdemeyer create