[Python-Dev] Tweak to PEP 523 for storing a tuple in co_extra (original) (raw)

Christian Heimes christian at python.org
Sat Sep 3 19:15:09 EDT 2016

On 2016-09-04 00:03, Yury Selivanov wrote:

On 2016-09-03 12:27 PM, Brett Cannon wrote: Below is the coextra section of PEP 523 with the update saying that users are expected to put a tuple in the field for easier simultaneous use of the field.

Since the coextra discussions do not affect CPython itself I'm planning on landing the changes stemming from the PEP probably on Monday. Tuples are immutable. If you have multiple coextra users then they will have to either mutate tuple (which isn't always possible, for instance, you can't increase size), or to replace it with another tuple. Creating lists is a bit more expensive, but item access speed should be in the same ballpark. Another question -- sorry if this was discussed before -- why do we want a PyObject* there at all? I.e. why don't we create a dedicated struct CoExtraContainer to manage the stuff in coextra? My understanding is that the users of coextra are C-level python optimizers and profilers, which don't need the overhead of CPython API. This way my work to add an extra caching layer (which I'm very much willing to continue to work on) wouldn't require another set of extra fields for code objects.

Quick idea before I go to bed:

You could adopt a similar API to OpenSSL's CRYPTO_get_ex_new_index() API, https://www.openssl.org/docs/manmaster/crypto/CRYPTO_get_ex_new_index.html

static int code_index = 0;

int PyCodeObject_NewIndex() { return code_index++; }

A library like Pyjion has to acquire an index first. In further calls it uses the index as offset into the new co_extra field. Libraries don't have to hard-code their offset and two libraries will never conflict. PyCode_New() can pre-populate co_extra with a PyTuple of size code_index. This avoids most resizes if you load Pyjion early. For code_index == 0 leaf the field NULL.

Christian

More information about the Python-Dev mailing list