[Python-Dev] Update on PEP 523 and adding a co_extra field to code objects (original) (raw)

Brett Cannon brett at python.org
Tue Aug 30 13:14:31 EDT 2016

On Tue, 30 Aug 2016 at 09:08 Antoine Pitrou <solipsis at pitrou.net> wrote:

On Mon, 29 Aug 2016 21:38:19 +0000 Brett Cannon <brett at python.org> wrote: > For quick background for those that don't remember, part of PEP 523 > proposed adding a coextra field to code objects along with making the > frame evaluation function pluggable ( > https://www.python.org/dev/peps/pep-0523/#expanding-pycodeobject). The idea > was that things like JITs and debuggers could use the field as a scratch > space of sorts to store data related to the code object. People who > objected to the new field did either for memory ("it adds another pointer > to the struct that won't be typically used"), or for conceptual reasons > ("the code object is immutable and you're proposing a mutable field"). The > latter is addressed by not exposing the field in Python and clearly stating > that code should never expect the field to be filled.

Just a question: Maciej mentioned the field may be useful for vmprof. That's already two potential users (vmprof and Pyjion) for a single field.

PTVS has also said they would find it useful for debugging.

OTOH, the PEP says:

"""It is not recommended that multiple users attempt to use the coextra simultaneously. While a dictionary could theoretically be set to the field and various users could use a key specific to the project, there is still the issue of key collisions as well as performance degradation from using a dictionary lookup on every frame evaluation. Users are expected to do a type check to make sure that the field has not been previously set by someone else.""" Does it mean that, in the event vmprof comes in and changes the field, Pyjion will have to recompile the function the next time it comes to execute it?

As of right now Pyjion simply doesn't JIT the function.

And, conversely, if Pyjion compiles a function while vmprof is enabled, vmprof will lose timing information (or whatever else, because I'm not sure what vmprof plans to store there) for that code object?

Depends on what vmprof chooses to do. Since the data is designed to be disposable it could decide it should always take precedence and overwrite the data if someone beat it to using the field. Basically I don't think we want co_extra1, co_extra2, etc. But we don't want to require a dict either as that kills performance. Using a list where users could push on objects might work, but I have no clue what that would do to perf as you would have to still needlessly search the list when only one piece of code uses the field.

Basically I don't see a good way to make a general solution for people who want to use the field simultaneously, so tools that use the field will need to be clear on how they choose to handle the situation, such "as we use it if it isn't set" or "we always use it no matter what". This isn't a perfect solution in all cases and I think that's just going to have to be the way it is. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20160830/377aa98d/attachment.html>

More information about the Python-Dev mailing list