[Python-Dev] DRAFT: python-dev summary for 2006-02-16 to 2006-02-28 (original) (raw)

Steven Bethard steven.bethard at gmail.com
Tue Apr 11 02:03:16 CEST 2006

Ok, here's the summary for the second half of February. Again, comments and corrections are greatly appreciated! (And thanks to those who already gave me some for the last summary.)

============= Announcements

Python release schedule

The Python 2.5 release schedule is PEP 356_. The first releases are planned for the end of March/beginning of April. Check the PEP for the full plan of features.

.. _PEP 356: http://www.python.org/dev/peps/pep-0356/

Contributing threads:

[SJB]

Buildbot improvements

Thanks to Benji York and Walter Dörwald, the buildbot results page_ now has a new CSS stylesheet that should make it a little easier to read. (And thanks to Josiah Carlson, we should now have a Windows buildbot slave.)

.. _buildbot results page: http://www.python.org/dev/buildbot/

Contributing threads:

[SJB]

Deprecation of multifile module

The multifile module, which has been supplanted by the email module since Python 2.2, is finally being deprecated. Though the module will not be removed in Python 2.5, its documentation now clearly indicates the deprecation.

Contributing thread:

[SJB]

Win64 AMD64 binaries available

Martin v. Löwis has made AMD64 binaries_ available for the current trunk's Python. If you're using an AMD64 machine (a.k.a. EM64T or x64), give 'em a whirl and see how they work.

.. _amd64 binaries: http://www.dcl.hpi.uni-potsdam.de/home/loewis/

Contributing thread:

[SJB]

Javascript to adopt Python iterators and generators

On a slightly off-topic note, Brendan Eich has blogged_ that the next version of Javascript will borrow iterators, generators and list comprehensions from Python. Nice to see that the Python plague is even spreading to other programming languages now. ;)

.. _blogged: http://weblogs.mozillazine.org/roadmap/archives/2006/02/

Contributing thread:

[SJB]

========= Summaries

A dict with a default value

Guido suggested a defaultdict type which would act like a dict, but produce a default value when getitem was called and no key existed. The intent was to simplify code examples like::

# a dict of lists
for x in y:
    d.setdefault(key, []).append(value)

# a dict of counts
for x in y:
    d[key] = d.get(key, 0) + 1

where the user clearly wants to associate a single default with the dict, but has no simple way to spell this. People quickly agreed that the default should be specified as a function so that using list as a default could create a dict of lists, and using int as a default could create a dict of counts.

Then the real thread began. Guido proposed adding an on_missing method to the dict API, which would be called whenever __getitem__ found that the requested key was not present in the dict. The on_missing method would look for a default_factory attribute, and try to call it if it was set, or raise a KeyError if it was not. This would allow e.g. dd.default_factory = list to make a dict object produce empty lists as default values, and del dd.default_factory to revert the dict object to the standard behavior.

However, a number of opponents worried that confusion would arise when basic dict promises (like that x in d implies that x in d.keys() and d[x] doesn't raise a KeyError) could be conditionally overridden by the existence of a default_factory attribute. Others worried about complicating the dict API with yet another method, especially one that was never meant to be called directly (only overridden in subclasses). Eventually, Guido was convinced that instead of modifying the builtin dict type, a new collections.defaultdict should be introduced.

Guido then defended keeping on_missing as a method of the dict type, noting that without on_missing any subclasses (e.g. collections.defaultdict) that wanted to override the behavior for missing keys would have to override __getitem__ and pay the penalty on every call instead of just the ones where the key wasn't present. In the patch committed to the Python trunk, on_missing was renamed to __missing__ and though no __missing__ method is defined for the dict type, if a subclass defines it, it will be called instead of raising the usual KeyError.

Contributing threads:

[SJB]

Encode and decode interface in Python 3.0

Jason Orendorff suggested that bytes.encode() and text.decode() (where text is the name of Python 3.0's str/unicode) should be removed in Python 3.0. Guido agreed, suggesting that Python 3.0 should have one of the following APIs for encoding and decoding:

There was a lot of discussion about how hard it was for beginners to figure out the current .encode() and .decode() methods, and Martin v. Löwis suggested that the behavior::

py> "Martin v. Löwis".encode("utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf6 in

position 11: ordinal not in range(128)

would be better if replaced by Guido's suggested behavior::

py> "Martin v. Löwis".encode("utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'str' object has no attribute 'encode'

since the user would immediately know that they had made a mistake by trying to encode a string. However, some people felt that this problem could be solved by simply changing the UnicodeDecodeError to something more informative like ValueError: utf8 can only encode unicode objects.

M.-A. Lemburg felt strongly that text and bytes objects should keep both .encode() and .decode() methods as simple interfaces to the registered codecs. Since the codecs system handles general encodings (not just text<->bytes encodings) he felt that .encode() and .decode() should be available on both bytes and text objects and should be able to return whatever type the encoding deems appropriate. Guido repeated one of his design guidelines: the return value of a function should not depend on the value of the arguments. Thus he would prefer that bytes.decode() only return text and text.decode() only return bytes, regardless of the encodings passed in. (He didn't seem to be commenting on the architecture of the codecs module however, just the architecture of the bytes and text types.)

Contributing threads:

[SJB]

Writable closures

Almann T. Goo was considering writing a PEP to allow write access to names in nested scopes. Currently, names in nested scopes can only be read, not written, so the following code fails with an UnboundLocalError::

def getinc(start=0):
    def incrementer(inc=1):
        start += inc
        return start
    return incrementer

Almann suggested introducing a new declaration, along the lines of global, to indicate that assignments to a name should be interpreted as assignments to the name in the nearest enclosing scope. Initially, he proposed the term use for this declaration, but most of the thread participants seemed to prefer outer, allowing the function above to be written as::

def getinc(start=0):
    def incrementer(inc=1):
        outer start
        start += inc
        return start
    return incrementer

A variety of syntactic variants achieving similar results were proposed, including a way to name a function's local namespace::

def getinc(start=0):
    namespace xxx
    def incrementer(inc=1):
        xxx.start += inc
        return xxx.start
    return incrementer

a way to indicate when a single use of a name should refer to the outer scope, based on the syntax for relative imports_::

def getinc(start=0):
    def incrementer(inc=1):
        .start += inc        # note the "."
        return .start        # note the "." (this one could be optional)
    return incrementer

and the previously suggested rebinding statement, from PEP 227_::

def getinc(start=0):
    def incrementer(inc=1):
        start +:= inc        # note the ":=" instead of "="
        return start
    return incrementer

Much like the last time this issue was brought up, "the discussion fizzled out after having failed to reach a consensus on an obviously right way to go about it" (Greg Ewing's quite appropriate wording). No PEP was produced, and it didn't seem like one would soon be forthcoming.

.. _PEP 227: http://www.python.org/peps/pep-0227.html .. _relative imports: http://www.python.org/peps/pep-0328.html

Contributing threads:

[SJB]

PEP 358: The "bytes" Object

This week mostly wrapped up the bytes type discussion from the last fortnight, with the introduction of PEP 358_: The "bytes" Object. The PEP proposes a bytes type which:

The bytes constructor allows an encoding for unicode objects (instead of requiring a call to unicode.encode) so as not to require double copying (one of encoding and one for conversion to bytes). Some people took issue with the fact that constructor allows an encoding for str objects, but ignores it, as this means code like bytes(s, 'utf-16be') will do a different thing for str and unicode. Ignoring the encoding argument for str objects was apparently intended to ease the transition from str to bytes, though it was not clear exactly how.

.. _PEP 358: http://www.python.org/peps/pep-0358.html

Contributing threads:

[SJB]

Compiling Python with MS VC++ 2005

M.-A. Lemburg suggested compiling Python with the new MS VC++ 2005, especially since it's "free". There was some concern about the stability of VS2005, and Benji York pointed out that the express editions are only free until November 6th. Fredrik Lundh pointed out that it would be substantially more work for all the developers who provide ready-made Windows binaries for multiple Python releases. In the end, they decided to keep with the current compiler at least for one more release.

.. _MS VC++ 2005: http://msdn.microsoft.com/vstudio/express/default.aspx .. _free until November 6th: http://msdn.microsoft.com/vstudio/express/support/faq/default.aspx#pricing

Contributing thread:

[SJB]

Alternate lambda syntax

Even though Guido already declared that Python 3.0 will keep the current lambda syntax, Talin decided to try out the new AST and give lambda a face-lift. With Talin's patch_, you can now write lambdas like::

>>> a = (x*x given (x))
>>> a(9)
81

>>> a = (x*y given (x=3,y=4))
>>> a(9, 10)
90
>>> a(9)
36
>>> a()
12

The patch was remarkably simple, and people were suitably impressed by the flexibility of the new AST. Of course, the patch was rejected since Guido is now happy with the current lambda situation.

.. _Talin's patch: http://bugs.python.org/1434008

Contributing thread:

[SJB]

Stateful codecs

Walter Dörwald was looking for ways to cleanly support stateful codecs. M.-A. Lemburg suggested extending the codec registry to maintain slots for the stateful encoders and decoders (and allowing six-tuples to be passed in) and adding the functions codecs.getencoderobject() and codecs.getdecoderobject(). Walter Dörwald suggested that codecs.lookup() should return objects with the following attributes:

(1) Name (2) Encoder function (3) Decoder function (4) Stateful encoder factory (5) Stateful decoder factory (6) Stream writer factory (7) Stream reader factory

For the sake of backwards compatibility, these objects would subclass tuple so that they look like the old four-tuples returned by codecs.lookup(). Walter's patch_ provides an implementation of some of these suggestions.

.. _Walter's patch: http://bugs.python.org/1436130

Contributing thread:

[SJB]

operator.is*Type and user-defined types

Michael Foord pointed out that for types written in Python, operator.isMappingType and operator.isSquenceType are essentially identical -- they both return True if __getitem__ is defined. Raymond Hettinger and Greg Ewing explained that for types written in C, these functions can give more detailed information because at the C level, CPython differentiates between the __getitem__ of the sequence protocol and the __getitem__ of the mapping protocol.

Contributing thread:

[SJB]

Python-level AST interface

Brett Cannon started a brief thread to discuss where to go next with the Python AST branch. Though some of the discussion moved online at PyCon, the major decisions were reported by Martin v. Löwis:

As a result, Python 2.5 now has a Python-level interface to AST objects::

>>> compile('"spam" if x else 42', '<string>', 'eval', 0x400)
<_ast.Expression object at 0x00BA0F50>

Contributing threads:

[SJB]

Allowing property to be used as a decorator

Georg Brandl suggested in passing that it would be nice if property() could be used as a decorator. Ian Bicking pointed out that you can already use property() this way as long as you only want a read-only property. However, the resulting property has no docstring, so Alex Martelli suggested that property use the doc of its fget no docstring was provided. Guido approved it, and Georg Brandl provided a patch_. Thus in Python 2.5, you'll be able to write read-only properties like::

@property
def x(self):
    """The x property"""
    return self._x + 42

.. _Georg Brandl provided a patch: http://bugs.python.org/1434038

Contributing threads:

[SJB]

Turning on unicode string literals for a module

Neil Schemenauer asked if it would be possible to have a from __future__ import unicode_strings statement which would turn all string literals into unicode literals for that module (without requiring the usual u prefix). Currently, you can turn on this kind of behavior for all modules using the undocumented -U command-line switch, but there's no way of enabling it on a per-module basis. There didn't seem to be enough momentum in the thread to implement such a thing however.

Contributing thread:

[SJB]

Allowing cProfile to print to other streams

Skip Montaro pointed out that the new cProfile module prints stuff to stdout. He suggested rewriting the necessary bits to add a stream= keyword argument where necessary and using stream.write(...) instead of the print statements. No patch was available at the time of this summary.

Contributing thread:

[SJB]

PEP 343 with-statement semantics

Mike Bland provided an initial implementation of PEP 343_'s with-statment. In writing some unit-tests for it, Guido discovered that the implementation would not allow generators like::

@contextmanager
def foo():
   try:
       yield
   except Exception:
       pass

with foo():
   1/0

to be equivalent to the corresponding in-line code::

try:
   1/0
except Exception:
   pass

because the PEP at the time did not allow context objects to suppress exceptions. Guido modified the patch and the PEP to require exit to reraise the exception if it didn't want it suppressed.

.. _PEP 343: http://www.python.org/peps/pep-0343.html

Contributing threads:

[SJB]

Dropping Win9x support in Python 2.6

Neal Norwitz suggested that Python 2.6 no longer try to support Win9x and WinME and updated PEP 11_ accordingly. There was a little rumbling about dropping the support, but no one stepped forward to volunteer to maintain the patches, and Guido suggested that anyone using a 6+ year old OS should be fine using an older Python too.

.. _PEP 11: http://www.python.org/dev/peps/pep-0011/

Contributing thread:

[SJB]

Removing non-Unicode support

Neal Norwitz suggested that the --disable-unicode switch might be a candidate for removal in Python 2.6. A few people were mildly concerned that the inability to remove Unicode support might make it harder to put Python on small hand-held devices. However, many (though not all) hand-helds already support Unicode, and currently a number of tests already fail if you use the --disable-unicode switch, so those who need this switch have not been actively maintaining it. Stripping out the numerous Py_USING_UNICODE declarations would substantially simplify some of the Python source. No final decision had been made at the time of this summary.

Contributing thread:

[SJB]

Translating the Python documentation

Facundo Batista had proposed translating the Library Reference and asked about how to get notifications when the documentation was updated (so that the translations could also be updated). Georg Brandl suggested a post-commit hook in SVN, though this would only give notifications at the module level. Fredrik Lundh suggested something based on his more dynamic library reference platform_ so that the notifications could indicate particular methods and functions instead.

.. _more dynamic library reference platform: http://effbot.org/zone/pyref.htm

Contributing threads:

[SJB]

PEP 338 updates

At Guido's suggestion, Nick Coghlan pared down PEP 338_ to just the bare bones necessary to properly implement the -m switch. That means the runpy module will contain only a single function, run_module, which will import the named module using the standard import mechanism, and then execute the code in that module.

.. _PEP 338: http://www.python.org/peps/pep-0338.html

Contributing thread:

[SJB]

Bugfix procedures

Just a reminder of the procedure for applying bug patches in Python (thanks to a brief thread started by Arkadiusz Miskiewicz). Anyone can submit a patch, but it will not be committed until a committer reviews and commits the patch. Non-committers are encouraged to review and comment on patches, and a number of the committers have promised that anyone who reviews and comments on at least five patches can have any patch they like looked at.

Contributing threads:

[SJB]

Removing --with-wctype-functions

M.-A. Lemburg suggested removing support for --with-wctype-functions as it makes Unicode support work in non-standard ways. Though he announced the plan in December 2004, PEP 11 wasn't updated, so removal will be delayed until Python 2.6.

Contributing thread:

[SJB]

Making ASCII the default encoding

Neal Norwitz asked if we should finally make ASCII the default encoding as PEP 263_ had promised in Python 2.3. He received only positive responses on this, and so in Python 2.5, any file missing a # -*- coding: ... -*- declaration and using non-ASCII characters will generate an error.

.. _PEP 263: http://www.python.org/peps/pep-0263.html

Contributing thread:

[SJB]

PEP 308: Conditional Expressions checked in

Thomas Wouters checked in a patch for PEP 308_, so Python 2.5 now has the long-awaited conditional expressions!

.. _PEP 308: http://www.python.org/dev/peps/pep-0308/

Contributing thread:

[SJB]

================== Previous Summaries

=============== Skipped Threads

More information about the Python-Dev mailing list