| msg29102 - (view) |
Author: Joe Brown (shmengie) |
Date: 2006-07-09 05:57 |
| While creating a huge zipfile (probably too big) the following error appeared. Exception exceptions.OverflowError: 'long int too large to convert to int' in <bound method ZipFile.__del__ of <zipfile.ZipFile instance at 0x009BD198>> ignored I assume this is during the closing of the zipfile, further assuming too many files were added to the archive. |
|
|
| msg29103 - (view) |
Author: Joe Brown (shmengie) |
Date: 2006-07-09 06:01 |
| Logged In: YES user_id=729883 Platform: Windows SBS 2003 Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on win32 |
|
|
| msg29104 - (view) |
Author: Ronald Oussoren (ronaldoussoren) *  |
Date: 2006-07-11 17:47 |
| Logged In: YES user_id=580910 How large is the resulting file? Zipfile in python2.4 cannot handle zipfiles that require the zip64 extensions and fails misserably when you do cross that limit. This was fixed in python 2.5. |
|
|
| msg29105 - (view) |
Author: Joe Brown (shmengie) |
Date: 2006-07-11 20:08 |
| Logged In: YES user_id=729883 The zip file was on order of 6-9 GBs, surely over 32000 files. I will give Python2.5 a shot and see how that works. I've stumbled on another bug with tarfile. It's not reporting an error, but I only find 350 files when I unarchive. That's entirely different bug tho :-o |
|
|
| msg29106 - (view) |
Author: Joe Brown (shmengie) |
Date: 2006-07-12 02:17 |
| Logged In: YES user_id=729883 Windows 2.5 beta 2 responded like this: Traceback (most recent call last): File "C:\python_scripts\archive.py", line 27, in main() File "C:\python_scripts\archive.py", line 23, in main zipper(zip, folder) File "C:\python_scripts\archive.py", line 15, in zipper zip.write(os.path.join(root, item)) File "C:\Python25\lib\zipfile.py", line 561, in write self._writecheck(zinfo) File "C:\Python25\lib\zipfile.py", line 536, in _writecheck raise LargeZipFile("Zipfile size would require ZIP64 extensions") zipfile.LargeZipFile: Zipfile size would require ZIP64 extensions C:\Python25\lib\struct.py:63: DeprecationWarning: struct integer overflow maskin g is deprecated return o.pack(*args) |
|
|
| msg29107 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2006-07-13 07:47 |
| Logged In: YES user_id=580910 The problem with zipfile is not the number of files, but the size of the archive. The basic zip file format stores file sizes as 32 bit (signed) integers, there is an extension for using 64 bit integers, but that isn't supported in python 2.4. |
|
|
| msg29108 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2006-07-13 08:17 |
| Logged In: YES user_id=580910 hmm, I should read all new comments before responding :-( To use the ZIP64 extensions with python2.5 you must add 'allowZip64=True' to arguments of the zipfile constructor. ZIP64 is disabled by default because the unix zip tools don't support zip64. |
|
|