(original) (raw)
On 6/24/06, Nick Coghlan <ncoghlan@gmail.com> wrote:
There are only two "different" possible callers for the whole API: a trusted interpreter or embedded application that launches an untrusted interpreter, and then there is *everyone* (this removes the distinction of trusted and untrusted interpreter and just views them as an interpreter). The former use the setting API to set the limits of the untrusted interpreter being created, while everyone else uses the API to make sure an untrusted interpreter does not overstep its bounds. The checks are done regardless of the type of interpreter, it just varies on whether the checks are NOOPs or not.
Since the memory cap seems to be causing the confusion, let me try it again. You are in a trusted interpreter or embedded app and you want an untrusted interpreter with a memory cap. You create the interpreter and call the PyXXX_SetMemoryCap() function on the untrusted interpreter to set that cap. Now, within the core interpreter code (untrusted or not) for where the interpreter allocates and deallocates memory there are calls to PyXXX_MemoryAllow() and PyXXX_MemoryFree(). If this is the trusted interpreter running, they are basically NOOPs. If it is an untrusted interpreter, then the actual checks are done to make sure the restriction is not broken.
And if an extension module has some reason to use the memory cap checks as well, it can. Same with an embedded application.
There is no grand distinction in terms of "this is for use in the core only while these are only for extension modules". It is just whether a function is used to set a restriction before an untrusted interpreter is used, or to check to make sure that a restriction is not violated.
-Brett
Brett Cannon wrote:
> Yep. That API will be used directly in the changes to pymalloc and
> PyMem_*() macros (or at least the basic idea). It is not *only* for
> extension modules but for the core as well.
>
> Existing extension modules and existing C code in the Python interpreter
> have no idea of any PyXXX_ calls, so I don't understand how new API
> functions help here.
>
>
> The calls get added to pymalloc and PyMem_*() under the hood, so that
> existing extension modules use the memory check automatically without a
> change. The calls are just there in case some one has some random need
> to do their own malloc but still want to participate in the cap. Plus
> it helped me think everything through by giving everything I would need
> to change internally an API.
This confused me a bit, too. It might help if you annotated each of the new
API's with who the expected callers were:
- trusted interpreter
- untrusted interpreter
- embedding application
- extension module
There are only two "different" possible callers for the whole API: a trusted interpreter or embedded application that launches an untrusted interpreter, and then there is *everyone* (this removes the distinction of trusted and untrusted interpreter and just views them as an interpreter). The former use the setting API to set the limits of the untrusted interpreter being created, while everyone else uses the API to make sure an untrusted interpreter does not overstep its bounds. The checks are done regardless of the type of interpreter, it just varies on whether the checks are NOOPs or not.
Since the memory cap seems to be causing the confusion, let me try it again. You are in a trusted interpreter or embedded app and you want an untrusted interpreter with a memory cap. You create the interpreter and call the PyXXX_SetMemoryCap() function on the untrusted interpreter to set that cap. Now, within the core interpreter code (untrusted or not) for where the interpreter allocates and deallocates memory there are calls to PyXXX_MemoryAllow() and PyXXX_MemoryFree(). If this is the trusted interpreter running, they are basically NOOPs. If it is an untrusted interpreter, then the actual checks are done to make sure the restriction is not broken.
And if an extension module has some reason to use the memory cap checks as well, it can. Same with an embedded application.
There is no grand distinction in terms of "this is for use in the core only while these are only for extension modules". It is just whether a function is used to set a restriction before an untrusted interpreter is used, or to check to make sure that a restriction is not violated.
-Brett