[Python-Dev] Last chance! (original) (raw)

Raymond Hettinger python at rcn.com
Sat Dec 20 13:47:56 EST 2003

> > I did notice at least one case where using ftstate might actually > > be a mistake: theoretically it's possible that two or more threads > > alternate calling next() on a generator (if they wrap it in a critical > > section); AFAICT the ftstate is never updated. > > Right, ftstate is never updated. > I think there is another inconsistent > situation, which can be created easily. > If a generator is run by a different thread > than it's creator, then the frame is run in that > other thread. evalfame correctly uses tstate, > but if tracing is activated, calltrace uses > frame->ftstate for no obvious reason, which > will probably mess up the tracing flags of the wrong > thread.

Right. Could you dig through CVS logs to find out when ftstate was first introduced? Maybe there's a clue about why there.

Here is a head start on the research.

The ceval.c use of tstate goes back to the introduction of generators in 2001.

The use in traceback.c and sysmodule.c goes back to 1997 when per-thread globals were factored into a structure to support separate thread-state management.

Prior to that, the history is more diffuse and harder to follow.

Raymond Hettinger

Ceval.c

2.252 (nascheme 21-Jun-01): PyThreadState *tstate =

2.252 (nascheme 21-Jun-01): * necessarily their creator. */ 2.255 (tim_one 23-Jun-01): Py_XINCREF(tstate->frame); 2.252 (nascheme 21-Jun-01): assert(f->f_back == NULL); 2.252 (nascheme 21-Jun-01): f->f_back = tstate->frame; 2.252 (nascheme 21-Jun-01):

revision 2.255 date: 2001/06/23 05:47:56; author: tim_one; state: Exp; lines: +2 -2 gen_iternext(): Don't assume that the current thread state's frame is not NULL. I don't think it can be NULL from Python code, but if using generators via the C API I expect a NULL frame is possible.

revision 2.252 date: 2001/06/21 02:41:10; author: nascheme; state: Exp; lines: +27 -14 Try to avoid creating reference cycles involving generators. Only keep a reference to f_back when its really needed. Do a little whitespace normalization as well. This whole file is a big war between tabs and spaces but now is probably not the time to reindent everything.

traceback.c

2.22 (guido 05-May-97): PyThreadState *tstate = frame->f_tstate; 2.22 (guido 05-May-97): tracebackobject *oldtb = (tracebackobjec

revision 2.22 date: 1997/05/05 20:56:15; author: guido; state: Exp; lines: +6 -30 Massive changes for separate thread state management. All per-thread globals are moved into a struct which is manipulated separately.

The story for their life a globals pre-dates what I can

sysmodule.c

2.45 (guido 02-Aug-97): PyThreadState *tstate = PyThreadState_Ge 2.45 (guido 02-Aug-97): PyObject *sd = tstate->interp->sysdict; 2.57 (guido 05-Oct-99): if (sd == NULL)

2.41 (guido 05-May-97): PyThreadState *tstate; 2.41 (guido 05-May-97): tstate = PyThreadState_Get(); 2.41 (guido 05-May-97): return Py_BuildValue( 2.41 (guido 05-May-97): "(OOO)", 2.41 (guido 05-May-97): tstate->exc_type != NULL ? tstat 2.41 (guido 05-May-97): tstate->exc_value != NULL ? tsta 2.41 (guido 05-May-97): tstate->exc_traceback != NULL ? 2.41 (guido 05-May-97): tstate->exc_traceback : 2.41 (guido 05-May-97): } 2.41 (guido 05-May-97):

revision 2.41 date: 1997/05/05 20:56:11; author: guido; state: Exp; lines: +30 -9 Massive changes for separate thread state management. All per-thread globals are moved into a struct which is manipulated separately.

Details of the 2.41 checkin

Index: sysmodule.c

RCS file: /cvsroot/python/python/dist/src/Python/sysmodule.c,v retrieving revision 2.40 retrieving revision 2.41 diff -c -r2.40 -r2.41 *** sysmodule.c 29 Apr 1997 20:42:30 -0000 2.40 --- sysmodule.c 5 May 1997 20:56:11 -0000 2.41 *************** *** 48,56 ****
#include "osdefs.h"
- PyObject *_PySys_TraceFunc, *_PySys_ProfileFunc; - int _PySys_CheckInterval = 10;

#if HAVE_UNISTD_H #include <unistd.h> #endif --- 48,53 ----

*** 98,103 **** --- 95,119 ---- }

static PyObject *

*** 112,123 **** PyObject *self; PyObject *args; { if (args == Py_None) args = NULL; else Py_XINCREF(args); ! Py_XDECREF(_PySys_TraceFunc); ! _PySys_TraceFunc = args; Py_INCREF(Py_None); return Py_None; } --- 128,140 ---- PyObject *self; PyObject *args; {

! Py_XDECREF(tstate->sys_tracefunc); ! tstate->sys_tracefunc = args; Py_INCREF(Py_None); return Py_None; }

*** 127,138 **** PyObject *self; PyObject *args; { if (args == Py_None) args = NULL; else Py_XINCREF(args); ! Py_XDECREF(_PySys_ProfileFunc); ! _PySys_ProfileFunc = args; Py_INCREF(Py_None); return Py_None; } --- 144,156 ---- PyObject *self; PyObject *args; {

! Py_XDECREF(tstate->sys_profilefunc); ! tstate->sys_profilefunc = args; Py_INCREF(Py_None); return Py_None; }

*** 142,148 **** PyObject *self; PyObject *args; { ! if (!PyArg_ParseTuple(args, "i", &_PySys_CheckInterval)) return NULL; Py_INCREF(Py_None); return Py_None; --- 160,167 ---- PyObject *self; PyObject *args; { ! PyThreadState *tstate = PyThreadState_Get(); ! if (!PyArg_ParseTuple(args, "i", &tstate->sys_checkinterval)) return NULL; Py_INCREF(Py_None); return Py_None;

*** 202,207 **** --- 221,227 ----

static PyMethodDef sys_methods[] = { /* Might as well keep this in alphabetic order */

*** 232,238 **** if (list == NULL) return NULL; for (i = 0; _PyImport_Inittab[i].name != NULL; i++) { ! PyObject *name = PyString_FromString(_PyImport_Inittab[i].name); if (name == NULL) break; PyList_Append(list, name); --- 252,259 ---- if (list == NULL) return NULL; for (i = 0; _PyImport_Inittab[i].name != NULL; i++) { ! PyObject *name = PyString_FromString( ! _PyImport_Inittab[i].name); if (name == NULL) break; PyList_Append(list, name);

More information about the Python-Dev mailing list