[Python-Dev] Hashes in Python3.5 for tuples and frozensets (original) (raw)

Anthony Flury anthony.flury at btinternet.com
Thu May 17 03:21:03 EDT 2018

Victor, Thanks for the link, but to be honest it will just confuse people - neither the link or the related bpo entries state that the fix is only limited to strings. They simply talk about hash randomization - which in my opinion implies ALL hash algorithms; which is why I asked the question.

I am not sure how much should be exposed about the scope of security fixes but you can understand my (and other's) confusion.

I am aware that applications shouldn't make assumptions about the value of any given hash value - apart from some simple assumptions based hash value equality (i.e. if two objects have different hash values they can't be the same value).

/B//TW : // // //This question was prompted by a question on a social media platform about the whether hash values are transferable between across platforms. Everything I could find stated that after Python 3.3 ALL hash values were randomized - but that clearly isn't the case; and the original questioner identified that some hash values are randomized and other aren't.// // //I did suggest strongly to the original questioner that relying on the same hash value across different platforms wasn't a clever solution - their original plan was to store hash values in a cross system database to enable quick retrieval of data (!!!). I did remind the OP that a hash value wasn't guaranteed to be unique anyway - and they might come across two different values with the same hash - and no way to distinguish between them if all they have is the hash. Hopefully their revised design will store the key, not the hash./

On 17/05/18 07:38, Victor Stinner wrote:

Hi,

String hash is randomized, but not the integer hash: $ python3.5 -c 'print(hash("abc"))' -8844814677999896014 $ python3.5 -c 'print(hash("abc"))' -7757160699952389646 $ python3.5 -c 'print(hash(1))' 1 $ python3.5 -c 'print(hash(1))' 1 frozenset hash is combined from values of the set. So it's only randomized if values hashes are randomized. The denial of service is more likely to occur with strings as keys, than with integers. See the following link for more information: http://python-security.readthedocs.io/vuln/cve-2012-1150hashdos.html Victor 2018-05-16 17:48 GMT-04:00 Anthony Flury via Python-Dev <python-dev at python.org>: This may be known but I wanted to ask this esteemed body first.

I understand that from Python3.3 there was a security fix to ensure that different python processes would generate different hash value for the same input - to prevent denial of service based on crafted hash conflicts. I opened two python REPLs on my Linux 64bit PC and did the following Terminal 1: >>> hash('Hello World') -1010252950208276719 >>> hash( frozenset({1,9}) ) -7625378979602737914 >>> hash(frozenset({300,301})) -8571255922896611313 >>> hash((1,9)) 3713081631926832981 >>> hash((875,932)) 3712694086932196356

Terminal 2: >>> hash('Hello World') -8267767374510285039 >>> hash( frozenset({1,9}) ) -7625378979602737914 >>> hash(frozenset({300,301})) -8571255922896611313 >>> hash((1,9)) 3713081631926832981 >>> hash((875,932)) 3712694086932196356 As can be seen - taking a hash of a string does indeed create a different value between the two processes (as expected). However the frozen set hash, the same in both cases, as is the hash of the tuples - suggesting that the vulnerability resolved in Python 3.3 wasn't resolved across all potentially hashable values. lI even used different large numbers to ensure that the integers weren't being interned. I can imagine that frozensets aren't used frequently as hash keys - but I would think that tuples are regularly used. Since that their hashes are not salted does the vulnerability still exist in some form ?. -- -- Anthony Flury email : Anthony.flury at btinternet.com Twitter : @TonyFlury <https://twitter.com/TonyFlury/>

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com

--

Anthony Flury email : Anthony.flury at btinternet.com Twitter : @TonyFlury <https://twitter.com/TonyFlury/>

More information about the Python-Dev mailing list