[Python-Dev] DRAFT: python-dev summary for 2006-09-16 to 2006-09-30 (original) (raw)

Steven Bethard steven.bethard at gmail.com
Sat Oct 28 00:23:50 CEST 2006

Thanks to all of those who have already given me feedback on the last summary. Here's the next one (for the second half of September). I found the "OS X universal binaries" and "Finer-grained locking than the GIL" discussions particularly hard to follow, so I'd especially appreciate corrections on those.

Thanks!

========= Summaries

Import features

Fabio Zadrozny ran into the previously reported relative import issues_ where a from . import xxx always fails from a top-level module. This is because relative imports rely on the __name__ of a module, so when it is just "__main__", they can't handle it properly.

On the subject of imports, Guido said that one of the missing import features was to be able to say "this package lives here". Paul Moore whipped up a Python API to an import hook that could do this, but indicated that a full mechanism would need to pay more attention to the environment (e.g. PYTHONPATH and .pth files).

There was also some discussion about trying to have a sort of per-module sys.path so that you could have multiple versions of the same module present, with different modules importing different versions. Phillip J. Eby suggested that this was probably not a very common need, and that implementing it would be quite difficult with things like C extensions only being able to be loaded once.

In general, people seemed interested in a pure-Python implementation of the import mechanism so that they could play with some of these approaches. It looked like Brett Cannon would probably be working on that.

.. _previously reported relative import issues: http://www.python.org/dev/summary/2006-06-16_2006-06-30/#relative-imports-and-pep-338-executing-modules-as-scripts

Contributing thread:

Python library documentation

A less-trolly-than-usual post from Xah Lee started a discussion about the Python documentation. Greg Ewing and others suggested following the documentation style of the Inside Macintosh series: first an "About this module" narrative explaining the concepts and how they fit together, followed by the extensive API reference. Most people agreed that simply extracting the documentation from the docstrings was a bad idea -- it lacks the high-level overview and gives equal importance to all functions, regardless of their use.

Contributing thread:

OS X universal binaries

Jack Howarth asked about creating universal binaries for OS X that would support 32-bit or 64-bit on both PPC and x86. Ronald Oussoren pointed out that the 32-bit part of this was already supported, but indicated that adding 64-bit support simultaneously might be more difficult. Ronald seemed to think that modifications to pyconfig.h.in might solve the problem, though he was worried that this might cause distutils to detect some architecture features incorrectly.

Contributing thread:

Finer-grained locking than the GIL

Martin Devera was looking into replacing the global interpreter lock (GIL) with finer-grained locking, tuned to minimize locking by assuming that most objects were used only by a single thread. For objects that were shared across multiple threads, this approach would allow non-blocking reads, but require all threads to "come home" before modifications could be made. Phillip J. Eby pointed out that most object accesses in Python are actually modifications too, due to reference counting, so it looked like Martin's proposal wouldn't work well with the current refcounting implementation of Python. After Martin v. Löwis found a bug in the locking algorithm, Martin Devera decided to take his idea back to the drawing board.

Contributing thread:

OS X and ssize_t formatting

The buildbots spotted an OS X error in the itertools module. After Jack Diederich fixed a bug where size_t had been used instead of ssize_t, Neal Norwitz noticed some problems with %zd on OS X. Despite documentation to the contrary in both the man page and the C99 Standard, using that specifier on OS X treats a negative number as an unsigned number. Ronald Oussoren and others reported the bug to Apple.

Contributing thread:

itertools.flatten()

Michael Foord asked about including a flatten function that would take a sequence with sub-sequences nested to an arbitrary depth and create a simple non-nested sequence from that. People were strongly opposed to adding this as a builtin, but even as an itertools function, there was disagreement. How should strings, dicts and other arbitrary iterables be flattened? Since there wasn't one clear answer, it looked like the proposal didn't have much of a chance.

Contributing thread:

Class definition syntax changes

Fabio Zadrozny noted that in Python 2.5, classes can now be declared as::

class C():
    ...

Some folks wanted the result to be a new-style class, but the presence or absence of () was deemed too subtle of a cue to make the new-style/old-style distinction. For the Python 2.X series, explicit subclassing of object will still be necessary.

Contributing thread:

Python 2.5 and GCC 4.2

Armin Rigo found some more signed integer overflows when using GCC 4.2 like the ones reported earlier_. Because Python 2.5 final was scheduled to be released in 24 hours, and it looked like there wouldn't be too many people affected these problems, they were deferred until 2.5.1. For the moment at least, the README indicates that GCC 4.1 and 4.2 shouldn't be used to compile Python.

.. _reported earlier: http://www.python.org/dev/summary/2006-08-16_2006-08-31/#gcc-4-2-and-integer-overflows

Contributing threads:

Discard method for dicts and lists

Gustavo Niemeyer and Greg Ewing suggested adding dict.discard() and list.discard() along the lines of set.discard(). Fred L. Drake, Jr. explained that dict.discard(foo) is currently supported with dict.pop(foo, None). There was more debate about the list version, but most people seemed to think that wrapping list.remove() with the appropriate if-statement or try-except was fine.

Contributing threads:

weakref enhancements

tomer filiba offered some additions to the weakref module, weakattr_ and weakmethod_. Raymond Hettinger questioned how frequently these would be useful in the real world, but both tomer and Alex Martelli assured him that they had real-world use-cases for these. However, there didn't generally seem to be enough support for them to include them in the standard library.

.. _weakattr: http://sebulba.wikispaces.com/recipe+weakattr .. _weakmethod: http://sebulba.wikispaces.com/recipe+weakmethod

Contributing thread:

AST structure guarantees

Anthony Baxter asked that the AST structure get the same guarantees as the byte-code format, that is, that it would change as little as possible so that people who wanted to hack it wouldn't have to change their code for each release. Pretty much everyone agreed that this was a good idea.

In a related thread, Sanghyeon Seo asked if the AST structure should become part of the Python specification so that other implementations like IronPython_ would use it as well. While most people felt like it would be good if the various specifications had similar AST representations, it seemed like mandating it as part of the implementation would lock things down too much.

.. _IronPython: http://www.codeplex.com/Wiki/View.aspx?ProjectName=IronPython

Contributing threads:

PEP 302: phase 2 import hooks

For his dissertation work, Brett Cannon needed to implement phase 2 of the PEP 302_ import hooks. He asked for feedback on whether it would be easier to do this within the current C code, or whether it would be better to rewrite the import mechanisms in Python first. Phillip J. Eby gave some advice on how to restructure things, and suggested that the C code was somewhat delicate and having a Python implementation around would be a Good Thing. Armin Rigo strongly recommended rewriting things in Python.

.. _PEP 302: http://www.python.org/dev/peps/pep-0302/

Contributing thread:

Testsuite fails on Windows if a space is in the path

Martin v. Löwis was trying to fix some bugs where spaces in Windows paths caused some of the testsuite to fail. For example, test_popen was getting an error because os.popen invoked::

cmd.exe /c "c:\Program Files\python25\python.exe" -c "import

sys;print sys.version"

which failed complaining that c:\Program is not a valid executable. Jean-Paul Calderon and Tim Peters explained that the cmd.exe part is necessary to force proper cmd.exe-style argument parsing and to allow environment variable substitution. After scrutinizing the MS quoting rules, it seemed like fixing this for Python 2.5 was too likely to introduce incompatibilities, so it was postponed to 2.6.

Contributing thread:

PEP 353: Backwards-compatibility #defines

David Abrahams suggested a modification to the suggested backwards-compatibility #define incantation of PEP 353_ so that the PY_SSIZE_T_MAX and PY_SSIZE_T_MIN would only ever get defined once. There was some discussion about whether or not this was absolutely necessary, but everyone agreed that the change was probably sensible regardless.

.. _PEP 353: http://www.python.org/dev/peps/pep-0353/

Contributing thread:

Bare-bones Python

Milan Krcmar asked about what he could drop from Python to make it small enough to fit on a platform with only 2 MiB of flash ROM and 16 MiB of RAM. Giovanni Bajo suggested dropping the CJK codecs (which account for about 800K), though he also noted that after that there weren't any really low-hanging fruit. Martin v. Löwis suggested that he might also get a gain out of dropping support for dynamic loading of extension modules, and linking all necessary modules statically. Gustavo Niemeyer pointed him to Python for S60_ and Python for Maemo_ which had to undergo similar stripping down.

.. _Python for S60: http://opensource.nokia.com/projects/pythonfors60/ .. _Python for Maemo: http://pymaemo.sf.net

Contributing thread:

================ Deferred Threads

================== Previous Summaries

=============== Skipped Threads

More information about the Python-Dev mailing list