[Python-checkins] python/nondist/peps pep-0000.txt, 1.271, 1.272 pep-0289.txt, 1.11, 1.12 pep-0320.txt, 1.10, 1.11 pep-0322.txt, 1.17, 1.18 pep-0329.txt, 1.4, 1.5 (original) (raw)

rhettinger at users.sourceforge.net rhettinger at users.sourceforge.net
Wed May 19 17🔞58 EDT 2004

Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5591

Modified Files: pep-0000.txt pep-0289.txt pep-0320.txt pep-0322.txt pep-0329.txt Log Message: Various PEP updates mostly related to generator expressions.

Index: pep-0000.txt

RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.271 retrieving revision 1.272 diff -C2 -d -r1.271 -r1.272 *** pep-0000.txt 20 Apr 2004 17:50:52 -0000 1.271 --- pep-0000.txt 19 May 2004 21🔞54 -0000 1.272

*** 63,67 **** S 252 Making Types Look More Like Classes GvR S 253 Subtyping Built-in Types GvR

*** 162,165 **** --- 161,165 ---- IF 283 Python 2.3 Release Schedule GvR SF 285 Adding a bool type GvR

*** 312,316 **** I 287 reStructuredText Docstring Format Goodger S 288 Generators Attributes and Exceptions Hettinger ! SA 289 Generator Expressions Hettinger I 290 Code Migration and Modernization Hettinger I 291 Backward Compatibility for Standard Library Norwitz --- 312,316 ---- I 287 reStructuredText Docstring Format Goodger S 288 Generators Attributes and Exceptions Hettinger ! SF 289 Generator Expressions Hettinger I 290 Code Migration and Modernization Hettinger I 291 Backward Compatibility for Standard Library Norwitz

Index: pep-0289.txt

RCS file: /cvsroot/python/python/nondist/peps/pep-0289.txt,v retrieving revision 1.11 retrieving revision 1.12 diff -C2 -d -r1.11 -r1.12 *** pep-0289.txt 28 Oct 2003 18:06:32 -0000 1.11 --- pep-0289.txt 19 May 2004 21🔞54 -0000 1.12 *************** *** 3,12 **** Version: RevisionRevisionRevision Last-Modified: DateDateDate ! Author: python at rcn.com (Raymond D. Hettinger) ! Status: Accepted Type: Standards Track Content-Type: text/x-rst Created: 30-Jan-2002 ! Python-Version: 2.3 Post-History: 22-Oct-2003
--- 3,12 ---- Version: RevisionRevisionRevision Last-Modified: DateDateDate ! Author: python at rcn.com (Raymond D. Hettinger) ! Status: Final Type: Standards Track Content-Type: text/x-rst Created: 30-Jan-2002 ! Python-Version: 2.4 Post-History: 22-Oct-2003
*************** *** 34,39 **** sum([xx for x in range(10)])
! Time, clarity, and memory are conserved by using an generator ! expession instead::
sum(xx for x in range(10)) --- 34,38 ---- sum([xx for x in range(10)])
! Memory is conserved by using an generator expession instead::
sum(xx for x in range(10)) *************** *** 71,74 **** --- 70,80 ---- application.
+ Early timings showed that generators had a significant performance + advantage over list comprehensions. However, the latter were highly + optimized for Py2.4 and now the performance is roughly comparable + for small to mid-sized data sets. As the data volumes grow larger, + generator expressions tend to perform better because they do not + exhaust cache memory and they allow Python to re-use objects between + iterations.
BDFL Pronouncements *************** *** 94,103 **** is equivalent to::
! def __gen(): ! for x in range(10): yield x2 ! g = __gen() print g.next()
2. The syntax requires that a generator expression always needs to be directly inside a set of parentheses and cannot have a comma on --- 100,126 ---- is equivalent to::
! def __gen(exp): ! for x in exp: yield x2 ! g = __gen(range(10)) print g.next()
+ Only the outermost for-expression is evaluated immediately, the other + expressions are deferred until the generator is run:: + + + g = (tgtexp for var1 in exp1 if exp2 for var2 in exp3 if exp4) + + is equivalent to:: + + def __gen(bound_exp): + for var1 in bound_exp: + if exp2: + for var2 in exp3: + if exp4: + yield tgtexp + g = __generator(exp1) + del _gen + 2. The syntax requires that a generator expression always needs to be directly inside a set of parentheses and cannot have a comma on *************** *** 111,118 **** changes to::
! atom: '(' [listmaker1] ')'
! where listmaker1 is almost the same as listmaker, but only ! allows a single test after 'for' ... 'in'.
b) The rule for arglist needs similar changes. --- 134,143 ---- changes to::
! atom: '(' [testlist_gexp] ')' !
! where testlist_gexp is almost the same as listmaker, but only ! allows a single test after 'for' ... 'in'::
! testlist_gexp: test ( gen_for | (',' test)* [','] )

b) The rule for arglist needs similar changes. *************** *** 134,137 **** --- 159,164 ---- cases you have to parenthesize it.
+ The exact details were checked in to Grammar/Grammar version 1.49. + 3. The loop variable (if it is a simple variable or a tuple of simple variables) is not exposed to the surrounding function. This *************** *** 148,175 **** print x # prints "hello", not "c"
! 4. All free variable bindings are captured at the time this function ! is defined, and passed into it using default argument values. For ! example:: ! ! x = 0 ! g = (x for c in "abc") # x is not the loop variable! ! x = 1 ! print g.next() # prints 0 (captured x), not 1 (current x) ! ! This behavior of free variables is almost always what you want when ! the generator expression is evaluated at a later point than its ! definition. In fact, to date, no examples have been found of code ! where it would be better to use the execution-time instead of the ! definition-time value of a free variable. ! ! Note that free variables aren't copied, only their binding is ! captured. They may still change if they are mutable, for example:: ! ! x = [] ! g = (x for c in "abc") ! x.append(1) ! print g.next() # prints [1], not [] ! ! 5. List comprehensions will remain unchanged. For example::
[x for x in S] # This is a list comprehension. --- 175,179 ---- print x # prints "hello", not "c"
! 4. List comprehensions will remain unchanged. For example::
[x for x in S] # This is a list comprehension. *************** *** 203,206 **** --- 207,249 ---- immediately surrounding scope.
+ Early Binding versus Late Binding + ================================= + + After much discussion, it was decided that the first (outermost) + for-expression should be evaluated immediately and that the remaining + expressions be evaluated when the generator is executed. + + Asked to summarize the reasoning for binding the first expression, + Guido offered [5]:: + + Consider sum(x for x in foo()). Now suppose there's a bug in foo() + that raises an exception, and a bug in sum() that raises an + exception before it starts iterating over its argument. Which + exception would you expect to see? I'd be surprised if the one in + sum() was raised rather the one in foo(), since the call to foo() + is part of the argument to sum(), and I expect arguments to be + processed before the function is called. + + OTOH, in sum(bar(x) for x in foo()), where sum() and foo() + are bugfree, but bar() raises an exception, we have no choice but + to delay the call to bar() until sum() starts iterating -- that's + part of the contract of generators. (They do nothing until their + next() method is first called.) + + Various use cases were proposed for binding all free variables when + the generator is defined. And some proponents felt that the resulting + expressions would be easier to understand and debug if bound immediately. + + However, Python takes a late binding approach to lambda expressions and + has no precedent for automatic, early binding. It was felt that + introducing a new paradigm would unnecessarily introduce complexity. + + After exploring many possibilities, a consensus emerged that binding + issues were hard to understand and that users should be strongly + encouraged to use generator expressions inside functions that consume + their arguments immediately. For more complex applications, full + generator definitions are always superior in terms of being obvious + about scope, lifetime, and binding [6]_. +
Reduction Functions *************** *** 208,215 ****
The utility of generator expressions is greatly enhanced when combined ! with reduction functions like sum(), min(), and max(). Separate ! proposals are forthcoming that recommend several new accumulation ! functions possibly including: product(), average(), alltrue(), ! anytrue(), nlargest(), nsmallest().

--- 251,257 ----
The utility of generator expressions is greatly enhanced when combined ! with reduction functions like sum(), min(), and max(). The sandbox ! contains a draft statistics module with several new accumulation ! functions including: product(), average(), nlargest(), and nsmallest().

*************** *** 227,237 **** strong arguments that they were a desirable thing to have.
- * Samuele Pedroni provided the example of late binding. Various - contributors have made arguments for and against late binding.

References --- 269,286 ---- strong arguments that they were a desirable thing to have.
* Phillip Eby suggested "iterator expressions" as the name.
* Subsequently, Tim Peters suggested the name "generator expressions".
+ * Armin Rigo, Tim Peters, Guido van Rossum, Samuele Pedroni, + Hye-Shik Chang and Raymond Hettinger teased out the issues surrounding + early versus late binding [5]. + + * Jiwon Seo single handedly implemented various versions of the proposal + including the final version loaded into CVS. Along the way, there + were periodic code reviews by Hye-Shik Chang and Raymond Hettinger. + Guido van Rossum made the key design decisions after comments from + Armin Rigo and newsgroup discussions. Raymond Hettinger provided + the test suite, documentation, tutorial, and examples [6].
References *************** *** 251,254 **** --- 300,310 ---- http://python.org/sf/795947
+ .. [5] Discussion over the relative merits of early versus late binding + http://mail.python.org/pipermail/python-dev/2004-April/044555.html + + .. [6] Patch discussion and alternative patches on Source Forge + http://www.python.org/sf/872326 + + Copyright

Index: pep-0320.txt

RCS file: /cvsroot/python/python/nondist/peps/pep-0320.txt,v retrieving revision 1.10 retrieving revision 1.11 diff -C2 -d -r1.10 -r1.11 *** pep-0320.txt 24 Mar 2004 03:10:55 -0000 1.10 --- pep-0320.txt 19 May 2004 21🔞55 -0000 1.11 *************** *** 38,44 **** Completed features for 2.4
! PEP 218 Builtin Set Objects
! PEP 322 Reverse Iteration
Encapsulate the decorate-sort-undecorate pattern in a keyword for --- 38,46 ---- Completed features for 2.4
! PEP 218 Builtin Set Objects. ! ! PEP 289 Generator expressions.
! PEP 322 Reverse Iteration.
Encapsulate the decorate-sort-undecorate pattern in a keyword for *************** *** 49,58 **** The itertools module has two new functions, tee() and groupby().
!
Planned features for 2.4
- PEP 289 Generator expressions.

  PEP 292 Simpler String Substitutions to be implemented as a module.

--- 51,58 ---- The itertools module has two new functions, tee() and groupby().

! Add a collections module with a deque() object.

Planned features for 2.4

  PEP 292 Simpler String Substitutions to be implemented as a module.

*** 72,84 **** Add a module for statistical and reduction functions: stddev, average, nlargest, nsmallest, product, etc. ! ! Add collections package: ! - deque/queue (suggested by Raymond) ! - BTrees (suggested by Tim) ! - ? bag (only if use cases established) ! - ? heap (with a min/max option flag, a key= option, ! and implemented as a fibonacci heap) ! ! Possibly add 'diff3' and 'patch' functions to the difflib module.

  Finish implementing the Distutils bdist_dpkg command.  (AMK)

--- 72,79 ---- Add a module for statistical and reduction functions: stddev, average, nlargest, nsmallest, product, etc. ! (There is a tested implementation in the sandbox but it ! may be held-off until Py2.5 because there doesn't not ! appear to be any user demand and the author is concerned ! that the module may not offer a sufficiently rich API).

  Finish implementing the Distutils bdist_dpkg command.  (AMK)

Index: pep-0322.txt

RCS file: /cvsroot/python/python/nondist/peps/pep-0322.txt,v retrieving revision 1.17 retrieving revision 1.18 diff -C2 -d -r1.17 -r1.18 *** pep-0322.txt 6 Nov 2003 12:22:37 -0000 1.17 --- pep-0322.txt 19 May 2004 21🔞55 -0000 1.18

*** 4,8 **** Last-Modified: DateDateDate Author: Raymond Hettinger <python at rcn.com> ! Status: Accepted Type: Standards Track Content-Type: text/x-rst --- 4,8 ---- Last-Modified: DateDateDate Author: Raymond Hettinger <python at rcn.com> ! Status: Final Type: Standards Track Content-Type: text/x-rst

Index: pep-0329.txt

RCS file: /cvsroot/python/python/nondist/peps/pep-0329.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -C2 -d -r1.4 -r1.5 *** pep-0329.txt 20 Apr 2004 17:51:11 -0000 1.4 --- pep-0329.txt 19 May 2004 21🔞55 -0000 1.5 *************** *** 12,21 ****

- Status - ======

--- 12,15 ----

*** 25,28 **** --- 19,37 ---- constants and to apply that function throughout the standard library.

More information about the Python-checkins mailing list