Issue 18085: Verifying refcounts.dat - Python tracker (original) (raw)

Created on 2013-05-28 19:22 by serhiy.storchaka, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
refcounts_const_char.patch serhiy.storchaka,2013-05-31 20:20 Add `const` to `char*` arguments. review
Pull Requests
URL Status Linked Edit
PR 11247 merged serhiy.storchaka,2018-12-19 20:01
PR 11258 merged miss-islington,2018-12-20 07:34
Messages (17)
msg190236 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-28 19:22
I found one error in Doc/data/refcounts.dat (see , second argument of PyObject_CallMethodObjArgs is `PyObject*`, not `char*`). Perhaps there are other errors. How this file was generated first? Is it possible to write a tool which will automatically verify at least signatures of functions?
msg190238 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-05-28 19:48
New changeset 0889ab0d0da1 by Serhiy Storchaka in branch '3.3': Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat. http://hg.python.org/cpython/rev/0889ab0d0da1 New changeset ef9d42b98a3d by Serhiy Storchaka in branch '2.7': Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat. http://hg.python.org/cpython/rev/ef9d42b98a3d New changeset 6d0fd113a2e4 by Serhiy Storchaka in branch 'default': Issue #18085: Fix PyObject_CallMethodObjArgs()'s entry in refcounts.dat. http://hg.python.org/cpython/rev/6d0fd113a2e4
msg190240 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2013-05-28 19:55
I'm a little surprised that still exists. The first version was generated manually.
msg190245 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-05-28 20:23
Wow, I didn't even know that file existed. It could actually help with some static analysis of the C code to verify that the refcounts are as expected w/o actually having to do inter-procedural analysis to figure out all of the refcount values. I wonder if Dave's gcc work could help either verify the file or benefit from it?
msg190246 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2013-05-28 20:26
As far as I can tell that file is still manually maintained. I see it mentioned in Doc/conf.py, but nowhere else. It shouldn't be hard to deal with manually, as the C API doesn't change that often. A verifier shouldn't be terribly difficult to write, as Python C source is pretty consistently structured. I'm not volunteering for the task though.
msg190250 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2013-05-28 20:52
Were I adding that today, I'd use a more verbose (but more standard) format, like configparser or JSON. If any further use is going to be made of it, that should be considered. Colon-delimited is a pretty fragile format.
msg190251 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2013-05-28 20:55
> Fred L. Drake, Jr. added the comment: > > Were I adding that today, I'd use a more verbose (but more standard) > format, like configparser or JSON. If any further use is going to be > made of it, that should be considered. Colon-delimited is a pretty > fragile format. Given the context, it should be okay until Python is rewritten in C++. :-) I don't know where this is consumed. Presumably by Sphinx somewhere.
msg190288 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2013-05-29 11:16
And is it necessary to list all functions there? Many functions share the same behavior: they don't change the ownership of PyObject* passed as argument, and return a new reference. Only document functions that don't conform to this rule, like PyTuple_SET_ITEM and PyImport_AddModule.
msg190320 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2013-05-29 16:46
> And is it necessary to list all functions there? Probably not. However, BITD, I was figuring this stuff out as I was going along by examining the source code and writing down what I found. No regard for common or special cases. S
msg190414 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-05-31 20:20
Since creating refcounts.dat many functions changed their argument's types from `char*` to `const char*`. Here is a patch which fixes mismatches (perhaps not all). refcounts.dat in 3.x contains PyInt_* functions which don't exist in 3.x.
msg192885 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-07-11 19:01
New changeset ffe24e3e7a2a by Serhiy Storchaka in branch '3.3': Issue #18085: Add missed const modifier for some entries in refcounts.dat. http://hg.python.org/cpython/rev/ffe24e3e7a2a New changeset 6587fd3d89ae by Serhiy Storchaka in branch 'default': Issue #18085: Add missed const modifier for some entries in refcounts.dat. http://hg.python.org/cpython/rev/6587fd3d89ae
msg266865 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-06-02 02:46
Can we now close this as a duplicate of issue 9755?
msg266868 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2016-06-02 03:07
I don't think this is a duplicate of issue 9755; this relates to verifying the data, and that revolves around possible process improvements. Whether this issue should be closed is tied to whether the file has been verified, as the issue title suggests. I don't know whether Serhiy verified everything when he made his changes or not; that's not explicit in the issue or commit comments.
msg266879 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-02 07:43
No, I hadn't verified all names. I'm sure there are API functions not mentioned in refcounts.dat. See also . It would be nice to have a tool that parses Python source headers and produce lists of all public names in different formats for manual updating python3.def, refcounts.dat, docs, etc.
msg332164 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-19 20:24
PR 11247 adds more functions and fixes some errors.
msg332190 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-20 07:34
New changeset 83dd4e87a62311cfea5fdd37e8a945b6b07bccee by Serhiy Storchaka in branch 'master': bpo-18085: Update refcounts.dat. (GH-11247) https://github.com/python/cpython/commit/83dd4e87a62311cfea5fdd37e8a945b6b07bccee
msg332192 - (view) Author: miss-islington (miss-islington) Date: 2018-12-20 07:43
New changeset 73fc14d1f8441aef5ee03be627c63e74a6d915d6 by Miss Islington (bot) in branch '3.7': bpo-18085: Update refcounts.dat. (GH-11247) https://github.com/python/cpython/commit/73fc14d1f8441aef5ee03be627c63e74a6d915d6
History
Date User Action Args
2022-04-11 14:57:46 admin set github: 62285
2018-12-20 08:14:48 serhiy.storchaka set status: open -> closedstage: patch review -> resolvedresolution: fixedversions: + Python 3.7, Python 3.8, - Python 2.7, Python 3.3, Python 3.4
2018-12-20 07:43:29 miss-islington set nosy: + miss-islingtonmessages: +
2018-12-20 07:34:10 miss-islington set pull_requests: + <pull%5Frequest10487>
2018-12-20 07:34:00 serhiy.storchaka set messages: +
2018-12-19 20:24:57 serhiy.storchaka set messages: +
2018-12-19 20:23:19 serhiy.storchaka set pull_requests: - <pull%5Frequest10478>
2018-12-19 20:23:05 serhiy.storchaka set pull_requests: - <pull%5Frequest10477>
2018-12-19 20:01:49 serhiy.storchaka set stage: patch reviewpull_requests: + <pull%5Frequest10478>
2018-12-19 20:01:47 serhiy.storchaka set stage: (no value)pull_requests: + <pull%5Frequest10477>
2018-12-19 20:01:42 serhiy.storchaka set stage: (no value)pull_requests: + <pull%5Frequest10476>
2016-06-02 07:43:49 serhiy.storchaka set messages: +
2016-06-02 03:07:53 fdrake set messages: +
2016-06-02 02:46:18 berker.peksag set nosy: + berker.peksagmessages: +
2013-07-17 07:36:13 serhiy.storchaka set priority: normal -> low
2013-07-11 19:01:30 python-dev set messages: +
2013-05-31 20:20:01 serhiy.storchaka set files: + refcounts_const_char.patchkeywords: + patchmessages: +
2013-05-29 16:46:46 skip.montanaro set messages: +
2013-05-29 11:16:24 amaury.forgeotdarc set nosy: + amaury.forgeotdarcmessages: +
2013-05-28 20:55:55 skip.montanaro set messages: +
2013-05-28 20:52:15 fdrake set messages: +
2013-05-28 20:26:36 skip.montanaro set messages: +
2013-05-28 20:23:07 brett.cannon set nosy: + dmalcolm, brett.cannonmessages: +
2013-05-28 19:55:16 fdrake set nosy: + fdrakemessages: +
2013-05-28 19:48:07 python-dev set nosy: + python-devmessages: +
2013-05-28 19:22:38 serhiy.storchaka create