| msg254647 - (view) |
Author: Dingyuan Wang (gumblex) * |
Date: 2015-11-14 09:25 |
| The _make_zipfile in shutil uses ZIP_DEFLATED compression by default, and the fix introduced by #24982 adds directory entries. In zipfile.ZipFile.write, directories is added as 0 file_size, 0 compress_size, regardless of the compression method. Deflate will compress an empty string as \x03\x00, thus the directory entries become incorrect. The command line interface of zipfile is correct. Shutil can be fixed as zipfile.main. As a directory entry with compression methods other than ZIP_STORED is meaningless, zipfile.write and (maybe) zipfile.writestr should always write a ZIP_STORED header for directory entries to avoid the above problem occuring by programming mistakes. |
|
|
| msg254648 - (view) |
Author: SilentGhost (SilentGhost) *  |
Date: 2015-11-14 09:28 |
| Can you provide a test? |
|
|
| msg254649 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2015-11-14 10:11 |
| Do you want to provide a patch? |
|
|
| msg254650 - (view) |
Author: Dingyuan Wang (gumblex) * |
Date: 2015-11-14 10:13 |
| $ mkdir foo; touch foo/a.txt; python3 -c "import shutil; shutil.make_archive('foo', 'zip', base_dir='foo')"; unzip -t foo.zip Archive: foo.zip testing: foo/ error: invalid compressed data to inflate testing: foo/a.txt OK At least one error was detected in foo.zip. (This affects 2.7, 3.4+) |
|
|
| msg254651 - (view) |
Author: Dingyuan Wang (gumblex) * |
Date: 2015-11-14 10:52 |
| My patch for this. |
|
|
| msg255054 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2015-11-21 12:19 |
| Could you add a test Dingyuan? You can take test_zipfile_vs_zip in test_shutil as an example. The patch changes both shutil and zipfile. Aren't only changes to zipfile needed? |
|
|
| msg255084 - (view) |
Author: Dingyuan Wang (gumblex) * |
Date: 2015-11-22 06:50 |
| Yes, patching zipfile is enough. I wrote a test using `unzip -t` to check the zip. ZipFile.testzip can't detect this kind of error because zlib.decompressobj(-15) will decode b'' to b'' without errors. |
|
|
| msg255097 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2015-11-22 12:56 |
| New changeset e7d1df56f70b by Serhiy Storchaka in branch '3.4': Issue #25624: ZipFile now always writes a ZIP_STORED header for directory https://hg.python.org/cpython/rev/e7d1df56f70b New changeset 0f8d426e0d0c by Serhiy Storchaka in branch '3.5': Issue #25624: ZipFile now always writes a ZIP_STORED header for directory https://hg.python.org/cpython/rev/0f8d426e0d0c New changeset 09ed44192995 by Serhiy Storchaka in branch 'default': Issue #25624: ZipFile now always writes a ZIP_STORED header for directory https://hg.python.org/cpython/rev/09ed44192995 New changeset de5582c569ff by Serhiy Storchaka in branch '2.7': Issue #25624: ZipFile now always writes a ZIP_STORED header for directory https://hg.python.org/cpython/rev/de5582c569ff |
|
|
| msg255098 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2015-11-22 13:01 |
| Thank you for your contribution Dingyuan. Benjamin, I think this fix should be in 2.7.11. It was a regression in 2.7.11 that had made shutil.make_archive() unusable with the zip format. |
|
|
| msg255124 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2015-11-23 03:05 |
| New changeset 43208b0f2535 by Benjamin Peterson in branch '2.7': Issue #25624: ZipFile now always writes a ZIP_STORED header for directory entries. Patch by Dingyuan Wang. https://hg.python.org/cpython/rev/43208b0f2535 |
|
|
| msg255128 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2015-11-23 06:29 |
| Thank you Benjamin. |
|
|
| msg255135 - (view) |
Author: Larry Hastings (larry) * |
Date: 2015-11-23 07:28 |
| This change didn't make it into 3.5.1. It will be released with 3.5.2. I've already updated the Misc/NEWS entry. |
|
|
| msg255137 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2015-11-23 08:02 |
| Larry, I think this patch is enough important to go into 3.5.1 final. It fixes a regression in common operation. |
|
|
| msg256054 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2015-12-07 08:43 |
| Larry, is this fix included in 3.5.1? |
|
|
| msg256720 - (view) |
Author: Larry Hastings (larry) * |
Date: 2015-12-19 07:59 |
| Like I said--no. |
|
|