Issue 31911: Use malloc_usable_size() in pymalloc for realloc (original) (raw)

Issue31911

Created on 2017-10-31 17:57 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (3)
msg305319 - (view)	Author: STINNER Victor (vstinner) *	Date: 2017-10-31 17:57
Objects/obmalloc.c contains an interesting comment: if (!address_in_range(p, pool)) { /* pymalloc is not managing this block. If nbytes <= SMALL_REQUEST_THRESHOLD, it's tempting to try to take over this block. However, if we do, we need to copy the valid data from the C-managed block to one of our blocks, and there's no portable way to know how much of the memory space starting at p is valid. As bug 1185883 pointed out the hard way, it's possible that the C-managed block is "at the end" of allocated VM space, so that a memory fault can occur if we try to copy nbytes bytes starting at p. Instead we punt: let C continue to manage this block. */ return 0; } See also bpo-1185883. We don't have to guess, it's possible to get the size of a memory block allocated by malloc() with: * malloc_usable_size(ptr): available at least on Linux * _msize(ptr): Windows Maybe we could add a "msize" field to PyMemAllocatorEx? See also bpo-18835 whic adds PyMem_AlignedAlloc() and so already modify (and rename PyMemAllocatorEx). See also bpo-31626, but I'm not sure that it would help for _PyMem_DebugRawRealloc().
msg306402 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2017-11-16 23:11
What would be the point of this? This would complicate the code with platform-specific code and it's not obvious that it would change runtime performance in any significant way.
msg306870 - (view)	Author: STINNER Victor (vstinner) *	Date: 2017-11-24 01:26
I agree. I don't think that it's worth it to overcompilates the API. I proposed this change since we may change the API to add PyMem_AlignedAlloc(). Since I just rejected bpo-18835, it's no more the case. I close this issue as rejected.