Issue 26690: PyUnicode_Decode breaks when Python / sqlite3 is built with sqlite 3.12.0 (original) (raw)

So I really don't know where the issue is in this one, because I don't know enough about the different bits.

Step 1: Save this C program to demo.c:

#include <Python.h>

static PyObject * unicode_thing(PyObject *self, PyObject *value) { char *str; Py_ssize_t len;

if (value == Py_None)
    Py_RETURN_NONE;

if (PyString_AsStringAndSize(value, &str, &len))
    return NULL;

return PyUnicode_Decode(str, len, "utf-8", "ignore"); }

static PyMethodDef UnicodeMethods[] = { {"unicode_thing", unicode_thing, METH_O, "do a unicode thing."}, {NULL, NULL, 0, NULL} /* Sentinel */ };

PyMODINIT_FUNC initdemo(void) { (void) Py_InitModule("demo", UnicodeMethods); }

Step 2: Build with a setup.py:

from distutils.core import setup, Extension

module1 = Extension('demo', sources = ['demo.c'])

setup (name = 'PackageName', version = '1.0', description = 'This is a demo package', ext_modules = [module1])

$ python setup.py build_ext

  1. Run with a normal Python that is not built against SQLite 3.12.0:

$ python Python 2.7.10 (default, Sep 8 2015, 17:20:17) [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import demo demo.unicode_thing('sql') u'sql'

  1. Now build Python 2.7.11 with SQLite 3.12.0 in the -I / -L paths. Run the same program With that Python:

$ /opt/Python-2.7.11-sqlite-3.12.0/bin/python Python 2.7.11 (default, Apr 4 2016, 14:20:47) [GCC 5.3.1 20151207 (Red Hat 5.3.1-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import demo demo.unicode_thing('sql') u's\x00q'

Somehow the presence of sqlite-3.12.0 in the build is breaking the Python interpreter. I think. I really don't know. The bad news is, this is the code for SQLAlchemy's unicode processor, and as SQlite 3.12.0 starts getting put in distros, the world is going to break. So this is kind of really hard to test, I don't understand it, and it's totally urgent. Any insights would be appreciated!