Issue 1112949: ioctl has problems on 64 bit machines (original) (raw)

fcntly.ioctl takes an int as the second argument. If the value passed is a large 32 bit quantity (0x80046601 for example - EXT2_IOC_GETFLAGS) then I get:

Traceback (most recent call last): File "CommSecure-CVS/Operations/checkSpace.py", line 73, in ? main(sys.argv[1:]) File "CommSecure-CVS/Operations/checkSpace.py", line 25, in main os.path.walk(file, doDirectory, total) File "/usr/lib64/python2.3/posixpath.py", line 282, in walk func(arg, top, names) File "CommSecure-CVS/Operations/checkSpace.py", line 61, in doDirectory flags = fcntl.ioctl(fd, EXT3_IOC_GETFLAGS, " ") OverflowError: signed integer is greater than maximum

My guess here is that the code is checking against 32 bit quantities rather than 64 bit when converting to the C data type?

Platform is Linux, Fedora Core 3 on AMD Opteron.

Logged In: YES user_id=6757

The problem seems to be that Python integers are "long int," while the type expected by ioctl(2) is "unsigned long int" (on AMD64, at least).

My hackish workaround is to coerce the ioctl number to a signed quantity:

op = struct.unpack('i', struct.pack('I', op))[0]

Once it gets into the C code, the signed quantity is then coerced back to unsigned long int by the ioctl call (after an intermediate stop as (signed) int).

Probably, the correct course of action here is to figure out what type ioctl(2) uses on a particular architecture and adjust the PyArgs_ParseTuple() call and the type of "op" accordingly.