Message 225881 - Python tracker (original) (raw)

I'm not 100% certain this is a bug yet, but I'm beginning to think it's likely.

On 64-bit Linux, I can't pass a struct like this:

struct S { uint8_t data[16]; };

...to a function declared like this:

void f(struct S);

From experimentation with various integer types and array sizes, it seems this causes an abort somewhere in libffi any time the array is between 9 and 16 bytes in size. If the array is smaller or larger than that, the calls work as expected.

I've asked about this here: http://stackoverflow.com/questions/25487928/is-this-the-correct-way-to-pass-a-struct-by-value-in-ctypes

Here's some test code:

sum.cpp

#include <cstdint>

using std::size_t;

struct ArrayStruct {
    // We'll define ARRAY_TYPE and ARRAY_SIZE on the
    // command-line when we compile.
    std::ARRAY_TYPE data[ARRAY_SIZE];
};

extern "C" int64_t sum(struct ArrayStruct array)
{
    int64_t acc=0;
    for (size_t i=0; i!=ARRAY_SIZE; ++i)
    {
        acc+=array.data[i];
    }
    return acc;
}

sum.py

import ctypes
import sys

def main():
    array_size = int(sys.argv[1])
    array_type = sys.argv[2]

    libsum = ctypes.cdll.LoadLibrary('./libsum.so')

    ArrType = getattr(ctypes, 'c_' + array_type) * array_size

    class MyStruct(ctypes.Structure):
        _fields_ = [("data", ArrType)]

    m=MyStruct()
    for i in range(array_size):
        m.data[i]=i

    print(libsum.sum(m))

if __name__ == '__main__':
    main()

Build/run

$ g++ -g -shared -Wall -fPIC sum.cpp -o libsum.so -std=c++11 -D ARRAY_SIZE=16 -D ARRAY_TYPE=uint8_t && python3 sum.py 16 uint8
Aborted (core dumped)

I poked around a little bit in gdb. It's aborting in libffi's "ffi_call" function: https://github.com/atgreen/libffi/blob/v3.0.13/src/x86/ffi64.c#L516

(gdb) bt #0 0x00007ffff782cf79 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ffff7830388 in __GI_abort () at abort.c:89 #2 0x00007ffff67134f5 in ffi_call (cif=0x7fffffffd7b0, fn=0x7ffff650c625 <sum(ArrayStruct)>, rvalue=0x7fffffffd6f0, avalue=0x7fffffffd6d0) at ../src/x86/ffi64.c:516 #3 0x00007ffff691fee3 in _ctypes_callproc () from /usr/lib/python3.4/lib-dynload/_ctypes.cpython-34m-x86_64-linux-gnu.so #4 0x00007ffff6920578 in ?? () from /usr/lib/python3.4/lib-dynload/_ctypes.cpython-34m-x86_64-linux-gnu.so #5 0x000000000043810a in PyObject_Call () #6 0x0000000000579f45 in PyEval_EvalFrameEx () #7 0x000000000057d3d3 in PyEval_EvalCodeEx () #8 0x000000000057bfaa in PyEval_EvalFrameEx () #9 0x000000000057d3d3 in PyEval_EvalCodeEx () #10 0x000000000060ba83 in PyRun_FileExFlags () #11 0x000000000060bc85 in PyRun_SimpleFileExFlags () #12 0x000000000060d3ac in Py_Main () #13 0x000000000041ec0d in main () (gdb) frame 2 #2 0x00007ffff67134f5 in ffi_call (cif=0x7fffffffd7b0, fn=0x7ffff650c625 <sum(ArrayStruct)>, rvalue=0x7fffffffd6f0, avalue=0x7fffffffd6d0) at ../src/x86/ffi64.c:516 516 abort(); (gdb) info args cif = 0x7fffffffd7b0 fn = 0x7ffff650c625 <sum(ArrayStruct)> rvalue = 0x7fffffffd6f0 avalue = 0x7fffffffd6d0 (gdb) info locals a = j = size = 8 n = classes = {X86_64_INTEGER_CLASS, X86_64_NO_CLASS, 4294956784, 32767} stack = 0x7fffffffd4f0 "" argp = 0x7fffffffd5a0 "\001" arg_types = 0x7fffffffd6b0 gprcount = 1 ssecount = ngpr = 1 nsse = 0 i = avn = ret_in_memory = reg_args = 0x7fffffffd4f0 (gdb) print *cif $2 = {abi = FFI_UNIX64, nargs = 1, arg_types = 0x7fffffffd6b0, rtype = 0x7ffff6b5e228, bytes = 0, flags = 10}

It looks like we're trying to pass the struct in two registers, which I think is what's supposed to happen, but something is going wrong with the second register. It aborted because it has class X86_64_NO_CLASS and that's not handled by the switch.

I don't know if this is a bug in libffi, or if ctypes is feeding it bad information, or if I'm feeding ctypes bad information. I hope this information is useful for anyone investigating.

I get the same abort in both Python 2.7.6 and 3.4.0.

I originally stumbled across this issue trying to use PySDL2:

http://pysdl2.readthedocs.org/en/rel_0_9_3/

I was trying to call SDL_JoystickGetGUIDString, which uses a similar struct-by-value call:

http://hg.libsdl.org/SDL/file/92ca74200ea5/include/SDL_joystick.h