[Python-3000] the future of the GIL (original) (raw)
tomer filiba tomerfiliba at gmail.com
Wed May 9 10:47:02 CEST 2007
- Previous message: [Python-3000] Build is broken (r55196)
- Next message: [Python-3000] the future of the GIL
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On 5/8/07, Thomas Heller wrote:
Wouldn't multiple interpreters (assuming the problems with them would be fixed) in the same process give the same benefit? A separate GIL for each one?
hmm, i find this idea quite interesting really:
builtin immutable objects such as None, small ints, non-heap types, and builtin functions, would become uncollectible by the GC. after all, we can't reclaim their memory anyway, so keeping the accounting info is just a waste of time. the PyObject_HEAD struct would grow an "ob_collectible" field, which would tell the GC to ignore these objects altogether. for efficiency reasons, Py_INCREF/DECREF would still change ob_refcount, only the GC will ignore it for uncollectible objects.
each thread would have a separate interpreter, and all APIs should grow an additional parameter that specifies the interpreter state to use.
for compatibility reasons, we can also have a dict-like object mapping between thread-ids to interpreter states. when you invoke an API, it would get the interpreter state from the currently executing thread id. maybe that could be defined as a macro over the real API function.
the builtin immutable objects would be shared between all instances of the interpreter. other then those, all other objects would be local to the interpreter that created them
extension modules would have to be changed to support per-interpreter initialization.
in order to communicate between interpreters, we would use some kind of IPC mechanism, to serialize access to objects. of course it would be much more efficient, as no context switches are required in the same process. this would make each thread basically as protected as a OS process, so no locks would be required.
in order to support the IPC, a new builtin type, Proxy, would be added to the language. it would be the only object that can hold a cross-reference to objects in different interpreters -- much like today's RPC libs -- only that wouldn't have to work over a socket.
if python would ever have a tracing GC, that would greatly simplify things. also, moving to an atomic incref/decref library could also help.
of course i'm not talking about adding that to py3k. it's too immature even for a pre-pep. but continuing to develop that idea more could be the means to removing the GIL, and finally having really parallel python scripts.
-tomer
- Previous message: [Python-3000] Build is broken (r55196)
- Next message: [Python-3000] the future of the GIL
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]