[Python-Dev] segfault (double free?) when '''-string crosses line (original) (raw)

Tim Peters tim.peters at gmail.com
Mon Apr 10 06:13:18 CEST 2006

[Neal Norwitz]

http://python.org/sf/1467512 fixes the problem for me on linux. It converts all the PyMem* APIs to PyObject* APIs. Assigned to Guido until he changes that. :-)

Thanks! I didn't take that route, instead not changing anything I didn't take the time to understand first. For example, it's a pure loss to change tok_new() to use PyObject_ (as the patch does), since a struct tok_state is far too big for obmalloc to handle (e.g., the indstack member alone is too big -- that's an array of 100 ints, so consumes at least 400 bytes by itself). Using PyObject_ here just wastes some time figuring out that tok_state is too big for it, and passes the calls on to the system malloc/free.

I thought about switching the readline thingies at the time, but quickly hit a wall: I have no idea where the source for

#ifdef __VMS extern char* vms__StdioReadline(FILE *sys_stdin, FILE *sys_stdout, char *prompt); #endif

may be (it's apparently not checked in, and Google couldn't find it either). Does anyone still work on the VMS port? If not, screw it ;-)

There are several more places in the core that should probably use PyObject* memory APIs since the alloced memory is small. 412 uses of PyMem* in /.c. Most of those are in modules where it is probably appropriate. But PyFutureFeatures could really use PyObject* given it's only 8 bytes. (Python/future.c and Python/compile.c).

If we allocate one of those per PyAST_Compile, I'm not sure the difference would be measurable ;-)

Modules/bsddb.c has a scary line: #define PyObjectDel PyMemDEL

That one's fine -- it's protected by an appropriate #ifdef:

#if PYTHON_API_VERSION <= 1007 /* 1.5 compatibility */ #define PyObject_New PyObject_NEW #define PyObject_Del PyMem_DEL #endif

The name "PyObject_Del" didn't exist in Pythons that old, so "something like that" is necessary to write C that works with all Pythons ever released.

More information about the Python-Dev mailing list