[Python-Dev] If aligned_alloc() is missing on your platform, please let us know. (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat Oct 28 01:43:47 EDT 2017

On 27 October 2017 at 18:17, Stefan Krah <stefan at bytereef.org> wrote:

Victor wrote a patch and would like to avoid adding a (probably unnecessary) emulation function. I agree with that.

So if any platform does not have some form of alignedalloc(), please speak up.

I think Victor's suggested strategy in https://bugs.python.org/issue18835#msg305122 of deferring emulation support to its own follow-up issue is a good one, since there are two distinct cases to consider:

  1. CPython's own support for platforms where we don't have a native aligned memory allocation API to call is covered by PEP 11, so if all current buildbots still work, then it will be up to the folks interested in a platform that doesn't offer aligned allocations to provide both a suitable emulation and a buildbot to test it on.

2.While all of the CPython-provided memory allocators will implement the new slots, the folks implementing their own custom allocators will need a defined upgrade path in the "Porting" section of the What's New guide. For that, an explicit error on 3.7+ that says "Configured custom allocator doesn't implement aligned memory allocations" is likely going to be easier to debug than subtle issues with the way an implicit emulation layer interacts with the other memory allocator slots.

To appropriately address 2, we need more info not about which platforms natively support aligned memory allocations, but rather from folks that actually are implementing their own custom allocators. It may be that making the low level cross-platform raw alligned allocators available as a public API (independently of the switchable allocator machinery) will be a more appropriate way of handling that case than providing an emulation layer in terms of the old slots.

That is, the suggested approach for custom allocator developers would be:

  1. Test against 3.7, get an error complaining the new slots aren't defined
  2. Set the new slots to the CPython provided cross-platform abstraction (if appropriate), or else wrap that abstraction layer as needed, or else implement your own aligned allocation routines

If the only reason for the custom allocator was to allow tracemalloc to track additional memory that wouldn't otherwise be handled through CPython's memory allocators, then consider switching to using PyTraceMalloc_Track()/PyTraceMalloc_Untrack() instead.

Either way, that transition strategy discussion shouldn't block making the feature itself available - it will be a lot easier to discuss transition enablement if potentially affected folks can download 3.7.0a3 and tell us what actually breaks, rather than asking them to speculate about what they think might break.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20171028/e0cf23b9/attachment.html>

More information about the Python-Dev mailing list