[Python-Dev] Fast Implementation for ZIP decryption (original) (raw)

CJ Kucera pez at apocalyptech.com
Wed Jul 28 16:01:15 CEST 2010

CJ Kucera wrote:

Hello list, resurrecting a rather old thread from here:

http://mail.python.org/pipermail/python-dev/2009-August/091450.html

... and one final update from me, mostly just so Google and the like will pick it up. I did actually end up packaging up something I called "czipfile," which is just the stock Python Lib/zipfile.py optimized in Cython, which provides very fast zipfile decryption. It's available here:

http://pypi.python.org/pypi/czipfile

So, anyone with a need for fast zipfile decryption can now head over that way.

FWIW, while playing around with things, I found that you can get some pretty nontrivial performance improvements in pure Python by unrolling the decryption loop. For instance, line 608 in Lib/zipfile.py used to be:

newdata = ''.join(map(self.decrypter, newdata))

Substituting the following results in some significant speed improvements (though still quite a lot slower than the C-based extension) -

key0 = self.decrypter.key0
key1 = self.decrypter.key1
key2 = self.decrypter.key2
crctable = self.decrypter.crctable
datalist = []
for c in newdata:
    k = key2 | 2
    cord = ord(c) ^ (((k * (k^1)) >> 8) & 255)
    datalist.append(chr(cord))
    key0 = ((key0 >> 8) & 0xffffff) ^ crctable[(key0 ^ cord) & 0xff]
    key1 = (key1 + (key0 & 255)) & 4294967295
    key1 = (key1 * 134775813 + 1) & 4294967295
    key2 = ((key2 >> 8) & 0xffffff) ^ crctable[(key2 ^ ((key1 >> 24) & 255)) & 0xff]
self.decrypter.key0 = key0
self.decrypter.key1 = key1
self.decrypter.key2 = key2
newdata = ''.join(datalist)

Anyone looking to speed up decryption who didn't want to absolutely depend on the user having czipfile installed might want to consider bundling their own modified version of Lib/zipfile.py with the above changes.

Okay, that's all. Enjoy!

-CJ

-- WOW: Flemmy | "The ships hung in the sky in much the same pez at apocalyptech.com | way that bricks don't." - Douglas Adams, 24.24.2.3171 | The Hitchhiker's Guide To The Galaxy

More information about the Python-Dev mailing list