[Python-Dev] genexps slow? (original) (raw)

Michael Hudson mwh at python.net
Mon Apr 5 06:58:15 EDT 2004

"Tim Peters" <tim.one at comcast.net> writes:

[Tim Peters]

Resuming a generator function is a lot cheaper than calling a function, but there's still a non-trivial amount of code to get in and out of evalframe() each time, which the listcomp version gets to skip. [Michael Hudson. This is something that occurred to me a while ago: how many opcodes does a typical invocation of evalframe actually execute? A little script told me that the median length of functions in Lib/*.py was 38 instructions (or 52 bytes) IIRC, but obviously a dynamic count is far more interesting. If the number is fairly small (and honestly, I have no idea), the set up and tear down code becomes much more significant than one might think. And whatever it is, I expect it will be significantly smaller for functions synthesized to implement generator expressions.

I expect invocations of most generators (not necessarily synthetic genexp ones) probably execute very few instructions -- ~5, maybe?

I didn't think much about the code to get out of evalframe. There's a twist there: the functions synthesized for generator expressions get to take the quicker-than-general "fastyield" exit path. Hmm! It looks like tests against WHYYIELD before the fastyield label can no longer succeed, but are still there to slow down non-generator cases.

Hmm. I think I agree with you. I also think the tests against WHY_YIELD in the code for END_FINALLY are unnecessary, as we (syntactically) don't allow yields when there are finally: statements in play. If I'm wrong about that, then you're wrong too :-) It's all a bit subtle, to be sure.

Once again, I wonder if there was someway we could have two eval_frames: one with tracing support, one without. Unfortunately, I don't think this can fly (how do you start tracing?). Maybe we could have a '-notrace' command line flag? But I suspect that this is a silly over-complication.

Cheers, mwh

-- "An infinite number of monkeys at an infinite number of keyboards could produce something like Usenet." "They could do a better job of it." -- the corollaries to Gene Spafford's Axiom #2 of Usenet

More information about the Python-Dev mailing list