Issue 25653: ctypes+callbacks+fork+selinux = crash (original) (raw)
Created on 2015-11-18 06:03 by arigo, last changed 2022-04-11 14:58 by admin.
Messages (6)
Author: Armin Rigo (arigo) * 
Date: 2015-11-18 06:03
Ctypes uses libffi's ffi_closure_alloc(), which has a bug that make existing applications obscurely crash in one situation: if we are running on SELinux, making use of callbacks, and forking. This is because ffi_closure_alloc() will detect that it is running on SELinux and use an alternative way to allocate memory for the callbacks.
It does that because selinux won't let a process mmap() any anonymous read-write-execute memory (at least in some settings; but libffi always uses the workaround if selinux is detected). The workaround is to create a temporary file and mmap() it twice (at randomly different addresses), once as a read-write mapping and once as a read-execute mapping. However, the internal structure of libffi requires that this mapping be MAP_SHARED (we can't easily first build the memory content, then write it to the temporary file and mmap() that fixed content in executable memory).
The problem with this is that if the process forks, this memory is shared. If one of the two processes then frees the callback, the memory becomes garbage in the other process.
The problem was reported a few times at various places already, but not in this bug tracker. See:
https://sourceware.org/ml/libffi-discuss/2009/msg00320.html
https://bugzilla.redhat.com/show_bug.cgi?id=531233
https://bugzilla.redhat.com/show_bug.cgi?id=707944
I am adding this issue to Python's bug tracker because, while in theory a libffi issue, it seems that Python is one of the very few libffi users that actually frees callbacks in this way. I don't have a solution for either libffi or ctypes, though. My own recommendation would be to stop using ffi_closure_alloc() and let the application either work (on selinux without deny_execmem) or cleanly trigger an error (on selinux with deny_execmem).
For reference, the issue was reported to CFFI's bug tracker about python-cryptography 1.0: it uses cffi's version of callbacks, whose implementation is close to ctypes', except not using ffi_closure_alloc() and so hitting the original selinux error instead of a crash. The file https://bitbucket.org/cffi/cffi/raw/default/c/malloc_closure.h inside CFFI comes from an older version of ctypes which (by chance in this case) does not call ffi_closure_alloc().
Author: Armin Rigo (arigo) *
Date: 2016-09-03 07:44
Attached trivial example. This gives for me a bus error when run with selinux (actually tested by changing the "return 0;" to "return 1;" in selinux_enabled_check() file Modules/_ctypes/libffi/src/closures.c).
If you comment out any of the two do_stuff() calls, everything works fine.
Author: Christian Heimes (christian.heimes) *
Date: 2016-09-03 10:07
Thanks Armin,
I didn't know that your reported the bug in bugs.python.org until today. Last year Armin and I spent a good amount of time to analyse the situation. Armin was able to come up with a different callback implementation for cffi that that does not use W/X memory mappings.
The problem affects mod_wsgi applications on SELinux systems (Fedora, CentOS, RHEL). For security reasons SELinux prevents Apache HTTPD to have writeable and executable memory pages. FFI callbacks with dynamic closures either require the fd workaround (which is buggy) or the application segfaults.
https://bugzilla.redhat.com/show_bug.cgi?id=1277224 https://bugzilla.redhat.com/show_bug.cgi?id=1337141 https://bugzilla.redhat.com/show_bug.cgi?id=1249685
Author: Armin Rigo (arigo) *
Date: 2016-09-03 16:01
For completeness:
the crasher I attached gets a bus error even before calling ffi_closure_free(). At that point, only ffi_closure_alloc() has been called---in both parent and child.
stricly speaking, cffi is not fixed: it has the same problem when using callbacks like ctypes. What Christian talks about is an alternative API that we came up with. It requires the user code to be slightly different, and is only available if using a C compiler is acceptable; it is not available in the ctypes-like mode.
Author: Petr Viktorin (petr.viktorin) *
Date: 2021-06-30 08:32
Here's a simpler reproducer.
Author: STINNER Victor (vstinner) *
Date: 2021-09-29 20:44
Another recent crash involving libffi, closure, fork and SELinux: https://bugzilla.redhat.com/show_bug.cgi?id=1977410 This bug comes from libffi, not from Python (but it can be easily reproducing using ctypes which uses libffi).
History
Date
User
Action
Args
2022-04-11 14:58:23
admin
set
github: 69839
2021-09-29 20:44:13
vstinner
set
nosy: + vstinner
messages: +
2021-06-30 08:32:42
petr.viktorin
set
files: + y.py
nosy: + petr.viktorin
messages: +
2021-03-05 15:40:25
eryksun
set
versions: + Python 3.8, Python 3.9, Python 3.10, - Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
2016-09-03 16:01:45
arigo
set
messages: +
2016-09-03 10:07:28
christian.heimes
set
messages: +
2016-09-03 09:55:09
christian.heimes
set
nosy: + christian.heimes
2016-09-03 07:45:16
arigo
set
versions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
2016-09-03 07:44:42
arigo
set
files: + x.py
messages: +
2015-11-18 09:28:50
eryksun
set
nosy: + eryksun
2015-11-18 06:03:04
arigo
create