| msg255159 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2015-11-23 13:57 |
| If ElementTree.iterparse() is called with file names, it opens a file. When resulting iterator is not exhausted, the file lefts not closed. >>> import xml.etree.ElementTree as ET >>> import gc >>> ET.iterparse('/dev/null') <xml.etree.ElementTree._IterParseIterator object at 0xb6f9e38c> >>> gc.collect() __main__:1: ResourceWarning: unclosed file <_io.BufferedReader name='/dev/null'> 34 Martin Panter proposed in to add an explicit way to clean it up, like a generator.close() method. |
|
|
| msg255164 - (view) |
Author: Anilyka Barry (abarry) *  |
Date: 2015-11-23 14:17 |
| I am unable to reproduce the issue on Windows 7 with 3.5.0; I have tried opening a small (non-empty) text. Here's the result: >>> import xml.etree.ElementTree as ET >>> import gc >>> ET.iterparse("E:/New.txt") <xml.etree.ElementTree._IterParseIterator object at 0x0023ABB0> >>> gc.collect() 59 |
|
|
| msg255171 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2015-11-23 14:56 |
| You have to enable deprecation warnings. Run the interpreter with the -Wa option. |
|
|
| msg255173 - (view) |
Author: Anilyka Barry (abarry) * |
Date: 2015-11-23 15:08 |
| Oh, my bad. Ignore my last message, behaviour is identical then. Thanks for clearing that up. |
|
|
| msg341003 - (view) |
Author: Stefan Behnel (scoder) * |
Date: 2019-04-27 17:32 |
| I don't think there is a need for a close() method. Instead, the iterator should close the file first thing when it's done with it, but only if it owns it. Therefore, the fix in issue 25688 seems correct. Closing can also be done explicitly in a finaliser of the iterator, if implicit closing via decref is too lax. |
|
|
| msg341253 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2019-05-02 08:01 |
| Implicit closing an exhausted iterator helps only the iterator is iterated to the end. If the iteration has been stopped before the end, we get a leak of the file descriptor. Closing the file descriptor in the finalizer can be deferred to undefined term, especially in implementations without reference counting. Since file descriptors are limited resource, this can cause troubles in real programs. Reasons for close() in iterparse objects are the same as for close in files and generators. Maybe we will need to implement the full generator protocol (send() and throw()) in the iterparse objects, but currently I do not know use cases for this. |
|
|
| msg341258 - (view) |
Author: Stefan Behnel (scoder) * |
Date: 2019-05-02 08:32 |
| Ok, I think it's reasonable to make the resource management explicit for the specific case of letting iterparse() open the file. That suggests that there should also be context manager support, given that safe usages would often involve a try-finally. Since it might not always be obvious for users when they need to close the iterator or not, I would also suggest to not let it raise an error on a double-close, i.e. if .close() was already called or the iterator was already exhausted (and the file closed automatically), calling .close() should just do nothing. |
|
|
| msg368308 - (view) |
Author: Furkan Onder (furkanonder) * |
Date: 2020-05-06 22:39 |
| Python 3.8.2 (default, Apr 8 2020, 14:31:25) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import xml.etree.ElementTree as ET >>> import gc >>> ET.iterparse('/dev/null') <xml.etree.ElementTree.iterparse..IterParseIterator object at 0x7fb96f679d00> >>> gc.collect() 34 The warning(__main__:1: ResourceWarning: unclosed file <_io.BufferedReader name='/dev/null'>) is no longer available in python3.8.2 |
|
|