[Python-Dev] ctypes: Configurable bitfield allocation strategy (original) (raw)

Vlad Riscutia riscutiavlad at gmail.com
Sat Jun 25 18:33:04 CEST 2011

I recently started looking at some ctypes issues. I dug a bit into http://bugs.python.org/issue6069 and then I found http://bugs.python.org/issue11920. They both boil down to the fact that bitfield allocation is up to the compiler, which is different in GCC and MSVC. Currently we have hard-coded allocation strategy based on paltform in cfields.c:

if (bitsize /* this is a bitfield request */

&& pfieldsize / we have a bitfield open */ #ifdef MSWIN32 /* MSVC, GCC with -mms-bitfields */ && dict->size * 8 == *pfieldsize #else /* GCC */ && dict->size * 8 <= *pfieldsize_ _#endif_ _&& (*pbitofs + bitsize) <= *pfieldsize) {_ _/* continue bit field */_ _fieldtype = CONTBITFIELD;_ _#ifndef MSWIN32_ _} else if (bitsize /* this is a bitfield request */_ _&& *pfieldsize /* we have a bitfield open */_ _&& dict->size * 8 >= *pfieldsize && (*pbitofs + bitsize) <= dict->size * 8) { /* expand bit field */ fieldtype = EXPANDBITFIELD; #endif

So when creating a bitfield structure, it's size can be different on Linux vs Windows.

class MyStructure(ctypes.BigEndianStructure): pack = 1 # aligned to 8 bits, not ctypes default of 32 fields = [ ("Data0", ctypes.c_uint32, 32), ("Data1", ctypes.c_uint8, 3), ("Data2", ctypes.c_uint16, 12), ]

sizeof for above structure is 6 on GCC build and 7 on MSVC build. This leads to some confusion and issues, because we can't always interop correctly with code compiled with different compiler than the one Python is compiled with on the platform. Short term solution is to add a warning in the documentation about this. Longer term though, I think it would be better to add a property on the Structure class for configurable allocation strategy, for example Native (default), GCC, MSVC and when allocating the bitfield, use given strategy. Native would behave similar to what happens now, but we would also allow GCC-style allocation on Windows for example and the ability to extend this if we ever run into similar issues with other compilers. This shouldn't be too difficult to implement, will be backwards compatible and it would improve interop. I would like to hear some opinions on this.

Thank you, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20110625/7b4f6fea/attachment.html>

More information about the Python-Dev mailing list