[Python-Dev] PEP 578: Python Runtime Audit Hooks (original) (raw)

Steve Dower steve.dower at python.org
Fri Mar 29 11:29:31 EDT 2019

On 29Mar2019 0334, Christian Heimes wrote:

On 28/03/2019 23.35, Steve Dower wrote:

Audit Hook ----------

In order to observe actions taken by the runtime (on behalf of the caller), an API is required to raise messages from within certain operations. These operations are typically deep within the Python runtime or standard library, such as dynamic code compilation, module imports, DNS resolution, or use of certain modules such as ctypes. The following new C APIs allow embedders and CPython implementors to send and receive audit hook messages:: # Add an auditing hook typedef int (*hookfunc)(const char *event, PyObject *args, void *userData); int PySysAddAuditHook(hookfunc hook, void *userData); # Raise an event with all auditing hooks int PySysAudit(const char *event, PyObject *args); # Internal API used during PyFinalize() - not publicly accessible void PyClearAuditHooks(void); The new Python APIs for receiving and raising audit hooks are:: # Add an auditing hook sys.addaudithook(hook: Callable[[str, tuple]]) # Raise an event with all auditing hooks sys.audit(str, *args)

Hooks are added by calling PySysAddAuditHook() from C at any time, including before PyInitialize(), or by calling sys.addaudithook() from Python code. Hooks cannot be removed or replaced. Hi Steve, I wonder if the hooks could be replaced by a more efficient mechanism. These days, Linux, macOS, and most recently Windows [1] support dtrace probes. DTrace is a very powerful and efficient mechanism to trace user-space processes from Kernel space. At least we should consider to add DTrace probes to the auditing framework. [1] https://techcommunity.microsoft.com/t5/Windows-Kernel-Internals/DTrace-on-Windows/ba-p/362902

Calling into those frameworks will still require as much work as these hooks do, and also make it very difficult to do things like auditing unit tests or using pure-Python code when you're not concerned about initialization (e.g. in a long-running web server).

So I'm inclined to say that if you want those probes, you can enable them by adding a hook that calls into them? A similar argument is made for using ETW on Windows (which will work on versions of Windows that have been released, unlike DTrace) (and yes, this is a real argument I've already had over this proposal ;) ), so I really think leaving it open-ended and Python-specific is the best approach.

(Reading further down the link you provided, it seems DTrace in Windows will only be enabled for essentially-administrators. So that rules it out as a substitute for this proposal in my opinion.)

Cheers, Steve

More information about the Python-Dev mailing list