(original) (raw)
On Tue, 30 Aug 2016 at 09:08 Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 29 Aug 2016 21:38:19 +0000
Brett Cannon <brett@python.org> wrote:
\> For quick background for those that don't remember, part of PEP 523
\> proposed adding a co\_extra field to code objects along with making the
\> frame evaluation function pluggable (
\> https://www.python.org/dev/peps/pep-0523/#expanding-pycodeobject). The idea
\> was that things like JITs and debuggers could use the field as a scratch
\> space of sorts to store data related to the code object. People who
\> objected to the new field did either for memory ("it adds another pointer
\> to the struct that won't be typically used"), or for conceptual reasons
\> ("the code object is immutable and you're proposing a mutable field"). The
\> latter is addressed by not exposing the field in Python and clearly stating
\> that code should never expect the field to be filled.
Just a question: Maciej mentioned the field may be useful for vmprof.
That's already two potential users (vmprof and Pyjion) for a single
field.
PTVS has also said they would find it useful for debugging.
OTOH, the PEP says:
"""It is not recommended that multiple users attempt to use the co\_extra
simultaneously. While a dictionary could theoretically be set to the
field and various users could use a key specific to the project, there
is still the issue of key collisions as well as performance degradation
from using a dictionary lookup on every frame evaluation. Users are
expected to do a type check to make sure that the field has not been
previously set by someone else."""
Does it mean that, in the event vmprof comes in and changes the field,
Pyjion will have to recompile the function the next time it comes to
execute it?
As of right now Pyjion simply doesn't JIT the function.
And, conversely, if Pyjion compiles a function while vmprof is enabled,
vmprof will lose timing information (or whatever else, because I'm not
sure what vmprof plans to store there) for that code object?
Depends on what vmprof chooses to do. Since the data is designed to be disposable it could decide it should always take precedence and overwrite the data if someone beat it to using the field. Basically I don't think we want co\_extra1, co\_extra2, etc. But we don't want to require a dict either as that kills performance. Using a list where users could push on objects might work, but I have no clue what that would do to perf as you would have to still needlessly search the list when only one piece of code uses the field.
Basically I don't see a good way to make a general solution for people who want to use the field simultaneously, so tools that use the field will need to be clear on how they choose to handle the situation, such "as we use it if it isn't set" or "we always use it no matter what". This isn't a perfect solution in all cases and I think that's just going to have to be the way it is.