| msg228423 - (view) |
Author: Ivan Pozdeev (Ivan.Pozdeev) * |
Date: 2014-10-04 01:54 |
| a LibraryLoader returns the same _FuncPtr object for a given function every time. This way, if two libraries set its attributes (most possibly, `argtypes') to incompatible values (both representing the same C-level entities), one of them will stop working. I've just discovered such a problem with `pyreadline' and `colorama' which both utilize `windll.kernel32.GetConsoleScreenBufferInfo'. One solution is to make LibraryLoader return a new object each time. Another (worse IMO) is to provide a clone function for _FuncPtr (it cannot be clones by `copy.copy()'). An example code: >>> import pyreadline >>> import pip._vendor.colorama Readline internal error Traceback (most recent call last): File "c:\python27\lib\site-packages\pyreadline\console\console.py", line 768, in hook_wrapper_23 res = ensure_str(readline_hook(prompt)) File "c:\python27\lib\site-packages\pyreadline\rlmain.py", line 569, in readline self.readline_setup(prompt) File "c:\python27\lib\site-packages\pyreadline\rlmain.py", line 565, in readline_setup self._print_prompt() File "c:\python27\lib\site-packages\pyreadline\rlmain.py", line 466, in _print_prompt x, y = c.pos() File "c:\python27\lib\site-packages\pyreadline\console\console.py", line 261, in pos self.GetConsoleScreenBufferInfo(self.hout, byref(info)) ArgumentError: argument 2: <type 'exceptions.TypeError'>: expected LP_CONSOLE_SCREEN_BUFFER_INFO instance instead of pointer to CONSOLE_SCREEN_BUFFER_INFO (the same error is printed continuously, on Ctrl-C, the interpreter crashes) |
|
|
| msg228425 - (view) |
Author: Ivan Pozdeev (Ivan.Pozdeev) * |
Date: 2014-10-04 02:24 |
| Another possible solution is to prohibit settings attributes of vanilla _FuncPtr objects, requiring a library to make a private copy to be able to do that. |
|
|
| msg228426 - (view) |
Author: Eryk Sun (eryksun) *  |
Date: 2014-10-04 02:57 |
| The ctypes global LibraryLoader instances are convenient for Windows scripts and applications. Actually what they cache is the library (a CDLL instance), which in turn caches function pointers. Python packages, on the other hand, shouldn't use these particular loaders. As you've experienced, doing so can lead to conflicting definitions for restype, argtypes, and errcheck. Instead create a private loader such as cdll = LibraryLoader(CDLL), or windll = LibraryLoader(WinDLL). Doing this would be pretty much pointless on non-Windows platforms. To gain the benefit of the cache you'd have to use subscripting such as cdll['libc.so.6']. It's simpler to use libc = CDLL('libc.so.6'). |
|
|
| msg228503 - (view) |
Author: R. David Murray (r.david.murray) *  |
Date: 2014-10-04 22:14 |
| How does this relate to issue 14201? That is, is the answer just "use getitem if you don't want caching"? (And apply the doc patch from that issue :) |
|
|
| msg228522 - (view) |
Author: Ivan Pozdeev (Ivan.Pozdeev) * |
Date: 2014-10-05 01:33 |
| @R. David Murray: haha, the reverse change that introduced this problem in the first place! issue 14201's problem is exactly why I was going to suggest to also make _FuncPtr's compare equal if they point to the same function. @eryksun: Packages do this because it's the natural thing to do - there's no apparent way to clone a pointer, there's not even a notion they _need_ to be cloned. https://docs.python.org/2/library/ctypes.html#loading-shared-libraries only documents __getattr__() and behavior of a LibraryLoader, not of a CDLL . Bottom line: ctypes currently leaves a trap for users to fall into. While Python's paradigm is "make right things easy, make wrong things hard". So, he optimal way seems to require to clone function pointers to set attributes. I'm going to add a guard attribute and a `clone' method to _FuncPtr. It's discussable whether to make cloned/altered pointers compare equal to the originals. |
|
|
| msg228525 - (view) |
Author: Eryk Sun (eryksun) * |
Date: 2014-10-05 01:50 |
| > How does this relate to issue 14201? That is, is the answer just > "use getitem if you don't want caching"? Unlike CDLL.__getitem__, LibraryLoader.__getitem__ does use the attribute cache. Why use a LibraryLoader if you don't want the cached libs and access to their cached function pointers? That's its main reason for existing. But really only in Windows, since most DLL names work as a attributes and Windows LoadLibrary appends .DLL. If you don't want cached libs, don't use a LibraryLoader. Just use CDLL('msvcr100'), WinDLL('kernel32'), etc. If you want cached libs without polluting ctypes.cdll or ctypes.windll, just create your own instance such as windll = ctypes.LibraryLoader(ctypes.WinDLL). |
|
|
| msg228535 - (view) |
Author: Eryk Sun (eryksun) * |
Date: 2014-10-05 03:29 |
| > Packages do this because it's the natural thing to do I guess the tutorial is channeling projects toward using the cdll/windll LibraryLoader instances on Windows. It even shows using cdll.LoadLibrary('libc.so.6') on Linux. That's equivalent to CDLL('libc.so.6'); I don't know why one would bother with cdll.LoadLibrary. > there's not even a notion they _need_ to be cloned. The ctypes reference has always explained how CDLL instances cache function pointers via __getattr__ and (formerly) __getitem__. The same section also documents that LibraryLoader.__getattr__ caches libraries. However, it's missing an explanation of LibraryLoader.__getitem__, which returns getattr(self, name), for use when the library name isn't a valid Python identifier. > there's no apparent way to clone a pointer You can use pointer casting or from_buffer_copy to create a new function pointer. It isn't a clone because it only uses the function pointer type, not the current value of restype, argtypes, and errcheck. But this may be all you need. For example: >>> from ctypes import * >>> libm = CDLL('libm.so.6') cast: >>> sin = cast(libm.sin, CFUNCTYPE(c_double, c_double)) >>> sin(3.14/2) 0.9999996829318346 >>> sin2 = cast(sin, type(sin)) >>> sin2.argtypes (<class 'ctypes.c_double'>,) >>> sin2.restype <class 'ctypes.c_double'> from_buffer_copy: >>> sin = CFUNCTYPE(c_double, c_double).from_buffer_copy(libm.sin) >>> sin(3.14/2) 0.9999996829318346 https://docs.python.org/3/library/ctypes.html#ctypes.cast https://docs.python.org/3/library/ctypes.html#function-prototypes |
|
|
| msg228538 - (view) |
Author: Ivan Pozdeev (Ivan.Pozdeev) * |
Date: 2014-10-05 04:17 |
| > If you want cached libs without polluting ctypes.cdll or ctypes.windll, just create your own instance such as windll = ctypes.LibraryLoader(ctypes.WinDLL). This one looks like the next best thing to the current state of affairs, requiring minimal change to existing code. `cast' appears to be the "right way" when saving individual _FuncPtr's in a local namespace but it's far from being obvious. If going this way, a prominent warning in https://docs.python.org/2.7/library/ctypes.html?highlight=ctypes#loading-dynamic-link-libraries with the way to go would probably suffice. |
|
|
| msg229872 - (view) |
Author: Ivan Pozdeev (native_api) * |
Date: 2014-10-23 13:21 |
| Here's the warnings patch. No sure if the `copy.copy' recipe is officially supported. |
|
|