Issue 3552: uuid - exception on uuid3/uuid5 (original) (raw)

The test suite breaks on the Lib/test/test_uuid.py, as of r65661. This is because uuid3 and uuid5 now raise exceptions.

TypeError: new() argument 1 must be bytes or read-only buffer, not bytearray

The problem is due to the changes in the way "s#" now expects a read-only buffer in PyArg_ParseTupleAndKeywords. (Which was changed in r65661).

A rundown of the problem:

Lib/uuid.py:553 (in uuid.uuid3): hash = md5(namespace.bytes + bytes(name, "utf-8")).digest()

namespace.bytes is a bytearray, so the argument to md5 is a bytearray.

Modules/md5module.c:517 (in _md5.md5.new): if (!PyArg_ParseTupleAndKeywords(args, kwdict, "|s#:new", kwlist,

Using s# now requires a read-only buffer, so this raises a TypeError.

The same goes for uuid5 (which calls _sha1.sha1, and has exactly the same problem).

The commit log for r65561 suggests changing some s# into s* (which allows readable buffers). I don't understand the ramifications here (some problem with threading), and when I made that change, it seg faulted, so I'll leave well enough alone. But for someone who knows more what they're doing, that may be a more "root-of-the-problem" fix.

In the meantime, I propose this simple patch to fix uuid: I think namespace.bytes should actually return a bytes, not a bytearray, so I'm modifying it to return a bytes.

Related issue: http://bugs.python.org/issue3139

Patch for r65675. Commit log:

Fixed TypeError raised by uuid.uuid3 and uuid.uuid5, by passing a bytearray to hash functions. Now namespace.bytes returns a bytes instead of a bytearray.

I couldn't reproduce the problem (and apparently, many of the buildbots can't, either). It depends on whether you have openssl available, i.e. whether hashlib can be built. I explicitly disabled use of OpenSSL on my system, and have now committed a fix as r65676.

So are you saying that if I had libopenssl (or whatever the name is) installed and linked with Python, it would bypass the use of _md5 and _sha1, and call the hash functions in libopenssl instead? And all the buildbots do have it linked?

That would indicate that the bots aren't testing the code in _md5 and _sha1 at all. Perhaps one should be made to?

So are you saying that if I had libopenssl (or whatever the name is) installed and linked with Python, it would bypass the use of _md5 and _sha1, and call the hash functions in libopenssl instead?

Correct. Those modules aren't even built then. See openssl_ver in setup.py.

And all the buildbots do have it linked?

Most of them, yes, to be able to test openssl. You would have to check each one individually to really know.

That would indicate that the bots aren't testing the code in _md5 and _sha1 at all. Perhaps one should be made to?

Perhaps. However, the buildbots can't test all combinations, anyway. People will report problems in other combinations quickly. Hopefully, they don't panic when they encounter a bug that only shows up on their system.