[Python-Dev] DRAFT: python-dev summary for 2006-03-01 to 2006-03-15 (original) (raw)

Steven Bethard steven.bethard at gmail.com
Tue Apr 11 02:50:51 CEST 2006

Ok, if I summarize any more of python-dev, my brain's going to explode. ;-)

Here's the summaries for the first half of March. Let me know what to fix!

============= Announcements

Webstats for python.org

Thomas Wouters set up webalizer on dinsdale.python.org and added webstats for all subsites of python.org:

Check 'em out if you're interested!

Contributing thread:

[SJB]

Python 2.5 release schedule

The first releases scheduled for Python 2.5 are quickly approaching. Check PEP 356_ for details, but the first alpha is due on April 1st.

.. _PEP 356: http://www.python.org/doc/peps/pep-0356/

Contributing thread:

[SJB]

Py3K branch

Guido has begun work on Py3K, starting a new branch to rip out some stuff like string exceptions and classic classes. He's trying to get a "feel" for what Python 3.0 will look like, hopefully before his keynote in OSCON.

Contributing thread:

[SJB]

Deprecated modules going away in Python 2.5

A number of deprecated modules will be removed in Python 2.5, including:

and a variety of things from lib-old. These modules have been deprecated for a while now, and will be pulled in the next Python release.

Contributing thread:

[SJB]

========= Summaries

Maintaining ctypes in SVN trunk

Thomas Heller put ctypes into the Python SVN repository, and with the help of perky, Neal Norwitz and Thomas Wouters, updated it to take advantage of the new ssize_t feature. Thomas Heller has now moved the "official" ctypes development to the Python SVN.

Contributing threads:

[SJB]

Windows buildbots

Josiah Carlson had been working on getting a buildbot slave running on a Windows box, but eventually gave up due to crashes caused by VS.NET. Tim Peters fought his way through the setup with a XP box, posting his lessons_ to the wiki, and Trent Mick managed to follow a similar route and setup a Win2K buildbot slave. Thanks to all who suffered through the config -- Windows buildbot coverage looks pretty good now!

.. _his lessons: http://wiki.python.org/moin/BuildbotOnWindows

Contributing threads:

[SJB]

Python 3.0: itr.next() or next(itr)

The end of last fortnight's defaultdict thread turned to discussing the fate of the iterator protocol's .next() method in Python 3.0. Greg Ewing argued that renaming .next() to .next() and introducing a builtin function next() would be more consistent with the other magic methods and also more future-proof, since the next() function could be modified if the protocol method needed to change. Raymond Hettinger was very strongly against this proposal, suggesting that trading a Python-level attribute lookup for a Python-level global lookup plus a C-level slot lookup was not a good tradeoff.

The discussion then spread out to other protocol method/function pairs -- e.g. len() and len() -- and Oleg Broytmann suggested that they could all be replaced with methods, thus saving a lookup and clearing out the builtin namespace. Neil Schemenauer and Michael Chermside argued against such a change, saying that the double-underscore pattern allows new special methods to be introduced without worrying about breaking user code, and that using functions for protocols forces developers to use the same names when the protocols are involved, while using methods could allow some developers to choose different names than others.

Guido indicated that he'd like to do some usability studies to determine whether methods or functions were more intuitive for the various protocols.

Contributing threads:

[SJB]

Python 3.0: base64 encoding

This fortnight continued discussion from the last as to whether the base64 encoding should produce unicode or bytes objects. The encoding is specified in RFC 2045_ as "designed to represent arbitrary sequences of octets" using "a 65-character subset of US-ASCII". Traditionally, base64 "encoding" goes from bytes to characters, and base64 "decoding" goes from characters to bytes. But this is the inverse of the usual unicode meanings, where "encoding" goes from characters to bytes, and where "decoding" goes from bytes to characters.

Thus some people felt that the recent proposal to have only bytes.decode(), which would produce unicode, and unicode.encode(), which would produce bytes, would be a major problem for encodings like base64 which used the opposite terminology. A variety of proposals ensued, including putting .encode() and .decode() on both bytes and strings, having encode() and decode() builtins, and various ways of putting encoding and decoding into the unicode and bytes constructors or classmethods.

No clear solution could be agreed upon at the time.

.. _RFC 2045: http://www.ietf.org/rfc/rfc2045.txt

Contributing thread:

[SJB]

Coverity scans of Python code

Ben Chelf of Coverity presented scan.coverity.com, which provides the results of some static source code analysis looking for a variety of code defects in a variety of open source projects, including Python. Full access to the reports is limited to core developers, but Neal Norwitz explained a bit what had been made available. The types of problems reported in Python include unintialized variables, resource leak, negative return values, using a NULL pointer, dead code, use after free and some other similar conditions. The reports provide information about what condition is violated and where, and according to Neal have been high quality and accurate, though of course there were some false positives. Generally, developers seemed quite happy with the reports, and a number of bugs have subsequently been fixed.

Contributing threads:

[SJB]

Speeding up lookups of builtins

Steven Elliott was looking into reducing the cost of looking up Python builtins. Two PEPs (PEP 280_ and PEP 329_) had already been proposed for similar purposes, but Steven felt these were biting off too much at once as they tried to optimize all global attribute lookups instead of just those of builtins. His proposal would replace the global-builtin lookup chain with an array that indexed the builtins. A fast check for builtin shadowing would be performed before using a builtin; if no shadowing existed, the builtin would simply be extracted from the array, and if shadowing was present, the longer lookup sequence would be followed.

Guido indicated that he would like to be able to assume that builtins are not shadowed in Python 3.0, but made no comment on the particular implementation strategy suggested. Steven Elliott promised a PEP, though it was not yet available at the time of this summary.

.. _PEP 280: http://www.python.org/doc/peps/pep-0280/ .. _PEP 329: http://www.python.org/doc/peps/pep-0329/

Contributing thread:

[SJB]

Requiring parentheses for conditional expressions

Upon seeing a recent checkin using conditional expressions, Jim Jewett suggested that parentheses should be required around all conditional expressions for the sake of readability. The usual syntax debate ensued, and in the end it looked like the most likely result was that PEP 8_ would be updated to suggest parentheses around the "test" part of the conditional expression if it contained any internal whitespace.

.. _PEP 8: http://www.python.org/doc/peps/pep-0008/

Contributing threads:

[SJB]

Exposing a global thread lock

Raymond Hettinger suggested exposing the global interpreter lock to allow code to temporarily suspend all thread switching. Guido was strongly against the idea as it was not portable to Jython or IronPython and it was likely to cause deadlocks. However, there was some support for it, and others indicated that the only way it could cause deadlocks is if locks were acquired within the sections where thread switching was disabled, and that even these could be avoided by having locks raise an exception if acquired in such a section. However, Michael Chermside explained that supporting such a thing in Jython and IronPython would really be impossible, and suggested that the functionality be made available in an extension module instead.

Raymond Hettinger then suggested modifying sys.setcheckinterval() to allow thread switching to be stopped, but Martin v. Löwis explained that even disabling this "release the GIL from time to time" setting would not disable thread switching as, for example, the file_write call inside a PRINT_* opcode releases the GIL around fwrite regardless.

Contributing thread:

[SJB]

Making quit and exit callable

Ian Bicking resucitated a previous proposal to make quit and exit callable objects with informative repr messages like license and help already have. Georg Brandl submitted a patch that makes quit and exit essentially synonymous with sys.exit but with better repr messages. The patch was accepted and should appear in Python 2.5.

Contributing thread:

[SJB]

Python 3.0: Using C++

Fredrik Lundh suggested that in Python 3.0 it might be useful to switch to C++ instead of C for Python's implementation. This would allow some additional type checking, some more reliable reference counting with local variable destructors and smart pointers, and "native" exception handling. However, it would likely make linking and writing extension modules more difficult as C++ does not interoperate with others as happily as C does. Stephen J. Turnbull suggested that it might also be worth considering following the route of XEmacs -- all code must compile without warnings under both C and C++.

No final decision was made, but for the moment it looks like Python will stick with the status quo.

Contributing thread:

[SJB]

A @decorator decorator

Georg Brandl presented a patch providing a ``decorator`` decorator_ that would transfer a function's name, doc and dict attributes to the wrapped function. Initially, he had placed it in a new decorator module, but a number of folks suggested that this module and the functional module -- which currently only contains partial() -- should be merged into a functools module. At the time of this summary, the patch had not been applied.

.. _patch providing a decorator decorator: http://bugs.python.org/1448297

Contributing thread:

[SJB]

Expanding the use of as

Georg Brandl proposed that since as is finally becoming a keyword, other statements might allow as to be used for binding a name like the import and with statements do now. Generally people thought that using as to name the while or if conditions was not really that useful, especially since the parallel to with statements was pretty weak -- with statements bind the result of the context manager's enter() call, not the context manager itself.

Contributing thread:

[SJB]

Relative imports in the stdlib

Guido made a checkin using the new relative import feature_ in a few places. Because using relative imports can cause custom import's to break (because they don't take the new optional fifth argument), Guido backed out the changes, and updated PEP 8_ to indicate that absolute imports were to be preferred over relative imports whenever possible.

.. _relative import feature: http://www.python.org/doc/peps/pep-0328/

Contributing threads:

[SJB]

Making staticmethod objects callable

Nicolas Fleury suggested that staticmethod objects should be made callable so that code like::

class A(object):
    @staticmethod
    def foo(): pass
    bar = foo()

would work instead of compaining that staticmethod objects are not callable. Generally, people seemed to feel that most uses of staticmethods were better expressed as module-level functions anyway, and so catering to odd uses of staticmethods was not something Python needed to do.

Contributing thread:

[SJB]

Adding a uuid module to the stdlib

Fredrik Lundh suggested adding Ka-Ping Yee's uuid module_ to the standard library. Most people were agreeable to the idea, but with other uuid implementations around, there was some discussion about the details. Phillip J. Eby suggested something more along the lines of PEAK's uuid module_, but no final decisions were made.

.. _adding Ka-Ping Yee's uuid module: http://bugs.python.org/1368955 .. _PEAK's uuid module: http://svn.eby-sarna.com/PEAK/src/peak/util/uuid.py?view=markup

Contributing thread:

[SJB]

A dict that takes key= callable

Neil Schemenauer suggested providing dict variants in the collections module that would use the ids of the objects instead of the objects themselves. Guido suggested that it would likely be more useful to design a dict variant that took a key= argument like list.sort() does, and apply that key function to all keys in the dict. That would make implementing Neil's id-dict almost trivial and support a variety of other use-cases, like case-insensitive dicts. Peoplke seemed quite supportive of the proposal, but no patch was available at the time of this summary.

Contributing thread:

[SJB]

Bug in future processing

Martin Maly found Guido's previously encountered bug_ that Python 2.2 through 2.4 allow some assignment statements before the __future__ import. Tim Peters correctly channeled Guido that this was a bug and would be fixed in Python 2.5.

.. _Guido's previously encountered bug: http://mail.python.org/pipermail/python-dev/2006-January/060247.html

Contributing thread:

[SJB]

Cleaning up string code

Chris Perkins noted that str.count() is substantially slower than unicode.count(). Ben Cartwright and others indicated that the source for these functions showed clearly that the unicode version had been better optimized. Fredrik Lundh and Armin Rigo both mentioned cleaning up the string code to avoid some of the duplication and potentially to merge the str and unicode implementations together. At the time of this summary, it didn't seem that any progress towards this goal had yet been made.

Contributing thread:

[SJB]

Freeing Python-allocated memory to the system

Tim Peters spent his PyCon time working on a patch by Evan Jones, originally discussed in January. The patch enables Python to free memory back to the operating system so that code like::

x = []
for i in xrange(1000000):
   x.append([])
del x[:]

does not continue to consume massive memory after the del statement. Tim gave python-devvers some time to review the patch for any speed or other issues, and then committed it.

.. _a patch by Evan Jones: http://bugs.python.org/1123430 .. _discussed in January: http://mail.python.org/pipermail/python-dev/2005-January/051255.html

Contributing thread:

[SJB]

Modifying the context manager exit API

After thinking things over, Guido decided that the context manager __exit__ method should be required to return True if it wants to suppress an exception. This addressed the main concerns about the previous API, that if __exit__ methods were required to reraise exceptions, a lot of __exit__ methods might end up with easily-missed bugs.

Contributing thread:

[SJB]

Python 3.0: default comparisons

In Python 3.0, Guido plans to ditch the default < <= > >= comparisons currently provided, and only provide == != where by default all objects compare as unequal.

Contributing thread:

[SJB]

================ Deferred Threads

================== Previous Summaries

=============== Skipped Threads

More information about the Python-Dev mailing list