PEP 3108 – Standard Library Reorganization | peps.python.org (original) (raw)

Author:

Brett Cannon

Status:

Final

Type:

Standards Track

Created:

01-Jan-2007

Python-Version:

3.0

Post-History:

28-Apr-2008

Table of Contents

Note

The merging of profile/cProfile as of Python 3.3 did not occur, and thus is considered abandoned (although it would be acceptable to do in the future).

Abstract

Just like the language itself, Python’s standard library (stdlib) has grown over the years to be very rich. But over time some modules have lost their need to be included with Python. There has also been an introduction of a naming convention for modules since Python’s inception that not all modules follow.

Python 3.0 presents a chance to remove modules that do not have long term usefulness. This chance also allows for the renaming of modules so that they follow the Python style guide. This PEP lists modules that should not be included in Python 3.0 or which need to be renamed.

Modules to Remove

Guido pronounced that “silly old stuff” is to be deleted from the stdlib for Py3K [7]. This is open-ended on purpose. Each module to be removed needs to have a justification as to why it should no longer be distributed with Python. This can range from the module being deprecated in Python 2.x to being for a platform that is no longer widely used.

This section of the PEP lists the various modules to be removed. Each subsection represents a different reason for modules to be removed. Each module must have a specific justification on top of being listed in a specific subsection so as to make sure only modules that truly deserve to be removed are in fact removed.

When a reason mentions how long it has been since a module has been “uniquely edited”, it is in reference to how long it has been since a checkin was done specifically for the module and not for a change that applied universally across the entire stdlib. If an edit time is not denoted as “unique” then it is the last time the file was edited, period.

Previously deprecated [done]

PEP 4 lists all modules that have been deprecated in the stdlib. The specified motivations mirror those listed inPEP 4. All modules listed in the PEP at the time of the first alpha release of Python 3.0 will be removed.

The entire contents of lib-old will also be removed. These modules have already been removed from being imported but are kept in the distribution for Python for users that rely upon the code.

Platform-specific with minimal use [done]

Python supports many platforms, some of which are not widely used or maintained. And on some of these platforms there are modules that have limited use to people on those platforms. Because of their limited usefulness it would be better to no longer burden the Python development team with their maintenance.

The modules mentioned below are documented. All undocumented modules for the specified platforms will also be removed.

IRIX

The IRIX operating system is no longer produced [13]. Removing all modules from the plat-irix[56] directory has been deemed reasonable because of this fact.

Mac-specific modules

The Mac-specific modules are not well-maintained (e.g., the bgen tool used to auto-generate many of the modules has never been updated to support UCS-4). It is also not Python’s place to maintain such a large amount of OS-specific modules. Thus all modules underLib/plat-mac and Mac are to be removed.

A stub module for proxy access will be provided for use by urllib.

Solaris

Hardly used [done]

Some platform-independent modules are rarely used. There are a number of possible explanations for this, including, ease of reimplementation, very small audience or lack of adherence to more modern standards.

Obsolete

Becoming obsolete signifies that either another module in the stdlib or a widely distributed third-party library provides a better solution for what the module is meant for.

Maintenance Burden

Over the years, certain modules have become a heavy burden upon python-dev to maintain. In situations like this, it is better for the module to be given to the community to maintain to free python-dev to focus more on language support and other modules in the standard library that do not take up an undue amount of time and effort.

Modules to Rename

Many modules existed in the stdlib before PEP 8 came into existence. This has led to some naming inconsistencies and namespace bloat that should be addressed.

PEP 8 violations [done]

PEP 8 specifies that modules “should have short, all-lowercase names” where “underscores can be used … if it improves readability”. The use of underscores is discouraged in package names. The following modules violate PEP 8 and are not somehow being renamed by being moved to a package.

Current Name Replacement Name
_winreg winreg
ConfigParser configparser
copy_reg copyreg
Queue queue
SocketServer socketserver

Merging C and Python implementations of the same interface

Several interfaces have both a Python and C implementation. While it is great to have a C implementation for speed with a Python implementation as fallback, there is no need to expose the two implementations independently in the stdlib. For Python 3.0 all interfaces with two implementations will be merged into a single public interface.

The C module is to be given a leading underscore to delineate the fact that it is not the reference implementation (the Python implementation is). This means that any semantic difference between the C and Python versions must be dealt with before Python 3.0 or else the C implementation will be removed until it can be fixed.

One interface that is not listed below is xml.etree.ElementTree. This is an externally maintained module and thus is not under the direct control of the Python development team for renaming. See Open Issues for a discussion on this.

No public, documented interface [done]

There are several modules in the stdlib that have no defined public interface. These modules exist as support code for other modules that are exposed. Because they are not meant to be used directly they should be renamed to reflect this fact.

Current Name Replacement Name
markupbase _markupbase

Poorly chosen names [done]

A few modules have names that were poorly chosen in hindsight. They should be renamed so as to prevent their bad name from perpetuating beyond the 2.x series.

Current Name Replacement Name
repr reprlib
test.test_support test.support

Grouping of modules [done]

As the stdlib has grown, several areas within it have expanded to include multiple modules (e.g., support for database files). It thus makes sense to group related modules into packages.

dbm package

Current Name Replacement Name
anydbm dbm.__init__ [1]
dbhash dbm.bsd
dbm dbm.ndbm
dumbdm dbm.dumb
gdbm dbm.gnu
whichdb dbm.__init__ [1]

html package

Current Name Replacement Name
HTMLParser html.parser
htmlentitydefs html.entities

http package

Current Name Replacement Name
httplib http.client
BaseHTTPServer http.server [2]
CGIHTTPServer http.server [2]
SimpleHTTPServer http.server [2]
Cookie http.cookies
cookielib http.cookiejar

tkinter package

Current Name Replacement Name
Dialog tkinter.dialog
FileDialog tkinter.filedialog [4]
FixTk tkinter._fix
ScrolledText tkinter.scrolledtext
SimpleDialog tkinter.simpledialog [5]
Tix tkinter.tix
Tkconstants tkinter.constants
Tkdnd tkinter.dnd
Tkinter tkinter.__init__
tkColorChooser tkinter.colorchooser
tkCommonDialog tkinter.commondialog
tkFileDialog tkinter.filedialog [4]
tkFont tkinter.font
tkMessageBox tkinter.messagebox
tkSimpleDialog tkinter.simpledialog [5]
turtle tkinter.turtle

urllib package

Originally this new package was to be named url, but because of the common use of the name as a variable, it has been deemed better to keep the name urllib and instead shift existing modules around into a new package.

Current Name Replacement Name
urllib2 urllib.request, urllib.error
urlparse urllib.parse
urllib urllib.parse, urllib.request, urllib.error [6]
robotparser urllib.robotparser

xmlrpc package

Current Name Replacement Name
xmlrpclib xmlrpc.client
DocXMLRPCServer xmlrpc.server [3]
SimpleXMLRPCServer xmlrpc.server [3]

Transition Plan

Issues

Issues related to this PEP:

For modules to be removed

For module removals, it is easiest to remove the module first in Python 3.0 to see where dependencies exist. This makes finding code that (possibly) requires the suppression of the DeprecationWarning easier.

In Python 3.0

  1. Remove the module.
  2. Remove related tests.
  3. Remove all documentation (typically the module’s documentation file and its entry in a file for the Library Reference).
  4. Edit Modules/Setup.dist and setup.py if needed.
  5. Run the regression test suite (using -uall); watch out for tests that are skipped because an import failed for the removed module.
  6. Check in the change (with an appropriate Misc/NEWS entry).
  7. Update this PEP noting that the 3.0 step is done.

In Python 2.6

  1. Add the following code to the deprecated module if it is implemented in Python as the first piece of executed code (adjusting the module name and the warnings import and needed):
    from warnings import warnpy3k
    warnpy3k("the XXX module has been removed in Python 3.0",
    stacklevel=2)

del warnpy3k
or the following if it is an extension module:
if (PyErr_WarnPy3k("the XXX module has been removed in "
"Python 3.0", 2) < 0)
return;
(the Python-Dev TextMate bundle, available from Misc/TextMate, contains a command that will generate all of this for you). 2. Update the documentation. For modules with their own documentation file, use the :deprecated: option with the module directive along with the deprecated directive, stating the deprecation is occurring in 2.6, but is for the module’s removal in 3.0.:
.. deprecated:: 2.6
The :mod:XXX module has been removed in Python 3.0.
For modules simply listed in a file (e.g., undoc.rst), use thewarning directive. 3. Add the module to the module deletion test in test_py3kwarn. 4. Suppress the warning in the module’s test code using
test.test_support.import_module(name, deprecated=True). 5. Check in the change w/ appropriate Misc/NEWS entry (blockthis checkin in py3k!). 6. Update this PEP noting that the 2.6 step is done.

Renaming of modules

Support in the 2to3 refactoring tool for renames will be used to help people transition to new module names[9]. Import statements will be rewritten so that only the import statement and none of the rest of the code needs to be touched. This will be accomplished by using the as keyword in import statements to bind in the module namespace to the old name while importing based on the new name (when the keyword is not already used, otherwise the reassigned name should be left alone and only the module that is imported needs to be changed). The fix_imports fixer is an example of how to approach this.

Python 3.0

  1. Update 2to3 in the sandbox to support the rename.
  2. Use svn move to rename the module.
  3. Update all import statements in the stdlib to use the new name (use 2to3’s fix_imports fixer for the easiest solution).
  4. Rename the module in its own documentation.
  5. Update all references in the documentation from the old name to the new name.
  6. Run regrtest.py -uall to verify the rename worked.
  7. Add an entry in Misc/NEWS.
  8. Commit the changes.

Python 2.6

  1. In the module’s documentation, add a note mentioning that the module is renamed in Python 3.0:
    .. note::
    The :mod:OLDNAME module has been renamed to :mod:NEWNAME in
    Python 3.0.
  2. Commit the documentation change.
  3. Block the revision in py3k.

Open Issues

Renaming of modules maintained outside of the stdlib

xml.etree.ElementTree not only does not meet PEP 8 naming standards but it also has an exposed C implementation. It is an externally maintained package, though PEP 360. A request will be made for the maintainer to change the name so that it matches PEP 8and hides the C implementation.

Rejected Ideas

Modules that were originally suggested for removal

Introducing a new top-level package

It has been suggested that the entire stdlib be placed within its own package. This PEP will not address this issue as it has its own design issues (naming, does it deserve special consideration in import semantics, etc.). Everything within this PEP can easily be handled if a new top-level package is introduced.

References

This document has been placed in the public domain.