Issue 24161: PyIter_Check returns false positive for objects of type instance (original) (raw)

Created on 2015-05-10 20:55 by behzad.nouri, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
spammodule.c behzad.nouri,2015-05-10 20:55
doc_iter_check.diff rhettinger,2015-05-11 03:11 Doc patch
Messages (5)
msg242866 - (view) Author: behzad nouri (behzad.nouri) Date: 2015-05-10 20:55
- python 2 only, not reproducible on python 3 Attached file makes an extension module which just returns PyIter_Check value on passed object. Calling the function with an object of type "instance" returns true, even though the object is not iterator: >>> import spam >>> class Foo: ... pass ... >>> foo = Foo() >>> type(foo) <type 'instance'> >>> spam.isiter(foo) # <<<< ?! 1 >>> next(foo) TypeError: instance has no next() method
msg242870 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-05-10 22:50
The PyIter_Check() macro in Include/abstract.h does a quick test to see whether the tp_iternext slot is null or marked as not implemented. That works for builtin types but not for user defined classes (heap types). Old-style instances, see Objects/classobject.c::instance_iternext(), all define iternext with code that attempts lookup and call to the next() method, and if not it is not found, raises the TypeError you are seeing. The conflict is that PyIter_Check() aims to be a fast check of a static slot entry while instance_iternext() aims for a dynamic call-it-and-see-if-it-works check much like PyObject_HasAttr() does. Since this code is very old (back to Python 2.2) and has been mostly harmless (as far as we know), one resolution would be to just document this as a known limitation of PyIter_Check(). Rather than using PyIter_Check(), extensions should just call next() on object and see whether it succeeds.
msg242874 - (view) Author: behzad nouri (behzad.nouri) Date: 2015-05-10 23:29
> That works for builtin types but not for user defined classes > Rather than using PyIter_Check(), extensions should just call next() on object and see whether it succeeds but then, what would be the use case of PyIter_Check outside of python core?
msg242878 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2015-05-11 01:33
> but then, what would be the use case of PyIter_Check > outside of python core? You could still use it anywhere. It will give a correct result in the cases of extension modules, builtin types, and new-style classes. It will give a false positive in the case of old-style classes. The latter case doesn't seem to be of much consequence (there is a still a TypeError raised when next() is called), so you just find out a bit later than you otherwise would (I believe that is why this is why we haven't gotten a bug report in the 13+ years this code has existed). The feature is imperfect, incomplete and not as useful as it could be. But this ship sailed a long time ago. It is far too late for redesign (and risking unintended breakage). FWIW, PyIter_Check() is used several times in the Python core: sqlite, cPickle, and iter(). In those examples, there seem to be no adverse consequences for the false positive because we still get a TypeError downstream when the actual call is made to next().
msg242904 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-05-11 17:22
New changeset 0f7795edca65 by Raymond Hettinger in branch '2.7': Issue #24161: Document that PyIter_Check() returns false positives for old-style instances. https://hg.python.org/cpython/rev/0f7795edca65
History
Date User Action Args
2022-04-11 14:58:16 admin set github: 68349
2015-05-11 17:24:48 rhettinger set status: open -> closedresolution: fixed
2015-05-11 17:22:35 python-dev set nosy: + python-devmessages: +
2015-05-11 03:11:00 rhettinger set files: + doc_iter_check.diffkeywords: + patch
2015-05-11 01:33:30 rhettinger set priority: normal -> lowassignee: docs@pythoncomponents: + Documentation, - Interpreter Corenosy: + docs@pythonmessages: + stage: needs patch
2015-05-10 23:29:26 behzad.nouri set messages: +
2015-05-10 22:50:11 rhettinger set nosy: + rhettingermessages: +
2015-05-10 20:55:34 behzad.nouri create