Issue 30347: itertools.groupby() can fail a C assert() (original) (raw)

Created on 2017-05-11 21:10 by arigo, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (8)
msg293517 - (view)	Author: Armin Rigo (arigo) *	Date: 2017-05-11 21:10
This triggers an assert() failure on debug-mode Python (or a leak in release Python): from itertools import groupby def f(n): print("enter:", n) if n == 5: list(b) print("leave:", n) return n != 6 for (k, b) in groupby(range(10), f): print(list(b)) With current trunk we get: python: ./Modules/itertoolsmodule.c:303: _grouper_next: Assertion `gbo->currkey == NULL' failed.
msg293519 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2017-05-11 22:19
Do you have a suggested fix?
msg293549 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2017-05-12 13:46
PR 1557 implements the simplest fix -- remove the assert and use Py_XSETREF() instead of simple assignment. Since the resulting code in _grouper_next() is identical to the code in groupby_next(), it was shared between these two functions.
msg293579 - (view)	Author: Raymond Hettinger (rhettinger) *	Date: 2017-05-12 21:56
At first glance, the suggested PR looks fine. Unless there is a rush, I would like to hold-off spend more time thinking about Issue 30346 before moving forward with this one.
msg302948 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2017-09-25 11:55
Hmm, the sole is not enough for fixing this crash. I don't know if there is a simpler fix based on , but since PR 1557 removes the code duplication I'm going to merge it in any case.
msg303062 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2017-09-26 18:47
New changeset c740e4fe8a9bc5815dc18c38d7f7600b128c3c51 by Serhiy Storchaka in branch 'master': bpo-30347: Stop crashes when concurrently iterate over itertools.groupby() iterators. (#1557) https://github.com/python/cpython/commit/c740e4fe8a9bc5815dc18c38d7f7600b128c3c51
msg303065 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2017-09-26 19:20
New changeset 69b2dc8637ba924d78f9869be592c5a545510f10 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6': [3.6] bpo-30347: Stop crashes when concurrently iterate over itertools.groupby() iterators. (GH-1557) (#3770) https://github.com/python/cpython/commit/69b2dc8637ba924d78f9869be592c5a545510f10
msg303071 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2017-09-26 20:15
New changeset d0b9dc33676bdad217d5074954c1b37d4ca54a10 by Serhiy Storchaka in branch '2.7': [2.7] bpo-30347: Stop crashes when concurrently iterate over itertools.groupby() iterators. (GH-1557). (#3772) https://github.com/python/cpython/commit/d0b9dc33676bdad217d5074954c1b37d4ca54a10