[Python-Dev] Python 2.7, long double vs allocator alignment, GCC 8 on x86-64 (original) (raw)

Benjamin Peterson benjamin at python.org
Wed Jan 31 02:20:18 EST 2018

Yes, changing obmalloc.c's alignment guarantees would definitely be the easiest solution. I think someone just needs to investigate whether it wastes a lot of memory.

On Tue, Jan 30, 2018, at 13:22, Gregory P. Smith wrote:

I'm curious if changing the obmalloc.c ALIGNMENT and ALIGNMENTSHIFT defines is sufficient to avoid ABI breakage.

-gps On Tue, Jan 30, 2018 at 1:20 PM Gregory P. Smith <greg at krypto.org> wrote: > The proper fix for this in the code would likely break ABI compatibility > (ie: not possible in python 2.7 or any other stable release). > > Clang's UBSAN (undefined behavior sanitizer) has been flagging this one > for a long time. > > In Python 3 a double is used instead of long double since 2012 as I did > some digging at the time: > https://github.com/python/cpython/commit/e348c8d154cf6342c79d627ebfe89dfe9de23817 > > -gps > > On Tue, Jan 30, 2018 at 10:59 AM Florian Weimer <fw at deneb.enyo.de> wrote: > >> I hope this is the right list for this kind of question. We recently >> tried to build Python 2.6 with GCC 8, and ran into this issue: >> >> <https://bugzilla.redhat.com/showbug.cgi?id=1540316> >> >> Also quoting for context: >> >> | PyInstanceNewRaw contains this code: >> | >> | inst = PyObjectGCNew(PyInstanceObject, &PyInstanceType); >> | if (inst == NULL) { >> | PyDECREF(dict); >> | return NULL; >> | } >> | inst->inweakreflist = NULL; >> | PyINCREF(klass); >> | inst->inclass = (PyClassObject *)klass; >> | inst->indict = dict; >> | PyObjectGCTRACK(inst); >> | >> | PyObjectGCTRACK expands to: >> | _>> | #define PyObjectGCTRACK(o) do { _ _>> | PyGCHead *g = PyASGC(o); _ _>> | if (g->gc.gcrefs != PyGCREFSUNTRACKED) _ _>> | PyFatalError("GC object already tracked"); _ >> | … >> | >> | Via: >> | >> | #define PyASGC(o) ((PyGCHead *)(o)-1) >> | >> | We get to this: >> | >> | /* GC information is stored BEFORE the object structure. */ >> | typedef union gchead { >> | struct { >> | union gchead *gcnext; >> | union gchead *gcprev; >> | Pyssizet gcrefs; >> | } gc; >> | long double dummy; /* force worst-case alignment */ >> | } PyGCHead; >> | >> | PyGCHead has 16-byte alignment. The net result is that >> | >> | PyObjectGCTRACK(inst); >> | >> | promises to the compiler that inst is properly aligned for the >> | PyGCHead type, but it is not: PyObjectGCNew returns a pointer which >> | is only 8-byte-aligned. >> | >> | Objects/obmalloc.c contains this: >> | >> | /* >> | * Alignment of addresses returned to the user. 8-bytes alignment works >> | * on most current architectures (with 32-bit or 64-bit address busses). >> | * The alignment value is also used for grouping small requests in size >> | * classes spaced ALIGNMENT bytes apart. >> | * >> | * You shouldn't change this unless you know what you are doing. >> | */ >> | #define ALIGNMENT 8 /* must be 2^N */ >> | #define ALIGNMENTSHIFT 3 >> | #define ALIGNMENTMASK (ALIGNMENT - 1) >> | >> | So either the allocator alignment needs to be increased, or the >> | PyGCHead alignment needs to be decreased. >> >> Is this a known issue? As far as I can see, it has not been fixed on >> the 2.7 branch. >> >> (Store merging is a relatively new GCC feature. Among other things, >> this means that on x86-64, for sufficiently aligned pointers, vector >> instructions are used to update multiple struct fields at once. These >> vector instructions can trigger alignment traps, similar to what >> happens on some other architectures for scalars.) _>> ________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org >> >

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/benjamin%40python.org

More information about the Python-Dev mailing list