[Python-Dev] Tweak to PEP 523 for storing a tuple in co_extra (original) (raw)

Brett Cannon brett at python.org
Sat Sep 3 21:21:21 EDT 2016

Great, thanks!

On Sat, Sep 3, 2016, 17:59 Guido van Rossum <gvanrossum at gmail.com> wrote:

Brett, I have not followed everything here but I have no problem with tweaks at this level as long as you are happy with it.

--Guido (mobile) On Sep 3, 2016 5:39 PM, "Brett Cannon" <brett at python.org> wrote:

On Sat, 3 Sep 2016 at 17:27 Yury Selivanov <yselivanov.ml at gmail.com> wrote:

On 2016-09-03 5:19 PM, Brett Cannon wrote: > > > On Sat, 3 Sep 2016 at 16:43 Yury Selivanov <yselivanov.ml at gmail.com_ _> <mailto:yselivanov.ml at gmail.com>> wrote: > > > > On 2016-09-03 4:15 PM, Christian Heimes wrote: > > On 2016-09-04 00:03, Yury Selivanov wrote: > >> > >> On 2016-09-03 12:27 PM, Brett Cannon wrote: > >>> Below is the coextra section of PEP 523 with the update > saying that > >>> users are expected to put a tuple in the field for easier > simultaneous > >>> use of the field. > >>> > >>> Since the coextra discussions do not affect CPython itself I'm > >>> planning on landing the changes stemming from the PEP probably > on Monday. > >> Tuples are immutable. If you have multiple coextra users then > they > >> will have to either mutate tuple (which isn't always possible, for > >> instance, you can't increase size), or to replace it with > another tuple. > >> > >> Creating lists is a bit more expensive, but item access speed > should be > >> in the same ballpark. > >> > >> Another question -- sorry if this was discussed before -- why > do we want > >> a PyObject* there at all? I.e. why don't we create a dedicated > struct > >> CoExtraContainer to manage the stuff in coextra? My > understanding is > >> that the users of coextra are C-level python optimizers and > profilers, > >> which don't need the overhead of CPython API. > > > As Chris pointed out in another email, the overhead is only in the > allocation, not the iteration/access if you use the PyTuple macros to > get the size and index into the tuple the overhead is negligible. Yes, my point was that it's as cheap to use a list as a tuple for coextra. If we decide to store PyObject in coextra. > >> > >> This way my work to add an extra caching layer (which I'm very much > >> willing to continue to work on) wouldn't require another set of > extra > >> fields for code objects. > > Quick idea before I go to bed: > > > > You could adopt a similar API to OpenSSL's CRYPTOgetexnewindex() > > API, > > > https://www.openssl.org/docs/manmaster/crypto/CRYPTOgetexnewindex.html > > > > > > static int codeindex = 0; > > > > int PyCodeObjectNewIndex() { > > return codeindex++; > > } > > > > A library like Pyjion has to acquire an index first. In further > calls it > > uses the index as offset into the new coextra field. Libraries > don't > > have to hard-code their offset and two libraries will never > conflict. > > PyCodeNew() can pre-populate coextra with a PyTuple of size > > codeindex. This avoids most resizes if you load Pyjion early. For > > codeindex == 0 leaf the field NULL. > > Sounds like a very good idea! > > > The problem with this is the pre-population. If you don't get your > index assigned before the very first code object is allocated then you > still have to manage the size of the tuple in coextra. So what this > would do is avoid the iteration but not the allocation overhead. > > If we open up the can of worms in terms of custom functions for this > (which I was trying to avoid), then you end up with Pyssizet > PyCodeExtraIndex(), PyObject * > PyCodeGetExtra(PyCodeObject *code, Pyssizet index), and int > PyCodeSetExtra(PyCodeObject *code, Pyssizet index, PyObject *data) > which does all the right things for creating or resizing the tuple as > necessary and which I think matches mostly what Nick had proposed > earlier. But the pseudo-code for PyCodeGetExtra() would be:: > > if coextra is None: > coextra = (None,) * nextextraindex; > return None > elif len(coextra) < index - 1:_ _> ... pad out tuple > return None > else: > return coextra[index] > > Is that going to save us enough to want to have a custom API for this? But without that new API (basically what Christian proposed) you'd need to iterate over the list in order to find the object that belongs to Pyjion. Yes. If we manage to implement my opcode caching idea, we'll have at least two known users of coextra. Without a way to claim a particular index in coextra you will have some overhead to locate your objects. Two things. One, I would want any new API to start with an underscore so people know we can and will change its semantics as necessary. Two, Guido would have to re-accept the PEP as this is a shift in the use of the field if this is how people want to go.

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20160904/b4035114/attachment-0001.html>

More information about the Python-Dev mailing list