[Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? (original) (raw)

Alexander Belopolsky alexander.belopolsky at gmail.com
Wed Jul 7 17:29:19 CEST 2010

On Tue, Jul 6, 2010 at 11:54 PM, Terry Reedy <tjreedy at udel.edu> wrote:

On 7/6/2010 3:59 PM, Alexander Belopolsky wrote:

I am more interested in Brett's overall vision than this particular module. I understand that to be one of a stdlib that is separate from CPython and is indeed the standard Python library.

I am also very much interested in the overall vision, but I would like to keep the datetime.py thread focused, so I a going to reply to broad questions under a separate subject.

Questions:

1. Would the other distributions use a standard stdlib rather than current individual versions?

I certainly hope they will. In the ideal word, passing test.regrtest with unmodified Lib should be the definition of what is called Python. I understand that there is already some work underway in this direction such as marking implementation specific tests with appropriate decorators.

2. Would the other distributions pool their currently separate stdlib efforts to help maintain one standard stdlib?

I believe that making stdlib and test.regrtest more friendly to alternative implementations will go long way towards this goal. It will, of course, be a decision that each project will have to make.

3. What version of Python would be allowed for use in the stdlib? I would like the stdlib for 3.x to be able to use 3.x code. This would be only a minor concern for CPython as long as 2.7 is maintained, but a major concern for the other implementation currently 'stuck' in 2.x only. A good 3to2 would be needed.

Availability of python equivalents will hopefully help "other implementation currently 'stuck' in 2.x only" to get "unstuck" and move to 3.x. I understand that this is a somewhat sensitive issue at the moment, but I believe a decision has been made supporting new features for 2.x is outside of python-dev focus.

4. Does not ctypes make it possible to replace a method of a Python-coded class with a faster C version, with something like  try:  connect to methods.dll  check that function xyx exists  replace Someclass.xyy with ctypes wrapper  except: pass For instance, the SequenceMatcher heuristic was added to speedup the matching process that I believe is encapsulated in one O(n**2) or so bottleneck method. I believe most everything else is O(n) bookkeeping. The ctypes modules is very CPython centric as far as I know. For the new modules, this may be a valid way to rapidly develop accelerated versions. For modules that are already written in C, I don't see much benefit in replacing them with ctypes wrappers.

[.. datetime specific discussion skipped ..] From scanning that and the posts here, it seems like a pep or other doc on dual version modules would be a good idea. It should at least document how to code the switch from python version to the x coded version and how to test both, as discussed. I am certainly not ready to write such PEP. I may be in a better position to contribute to it after I gain more experience with test_datetime.py. At the moment I have more questions than answers.

For example, the established practice appears to be:

modulename.py

Python code

try: from _modulename import * except: pass

This is supposed to generate a .pyc file with no python definitions in it if _modulename is available. The problem with datetime.py is that it have several helper methods like _ymd2ord() that will still stay in the module. Should an "else:" clause be added to clean these up? should these methods become class or static methods as appropriate?

The established practice for testing is

py_module = support.import_fresh_module('modulename', blocked=['_modulename']) c_module = support.import_fresh_module('modulename', fresh=['_modulename'])

class TestDefnitions: # not a unittest.TestCase subclass def test_foo(self): self.module.foo(..) ...

class C_Test(TestDefnitions, unittest.TestCase): module = c_module

class Py_Test(TestDefnitions, unittest.TestCase): module = py_module

For datetime.py this approach presents several problems:

  1. replacing datetime with self.module.datetime everywhere can get messy quickly.
  2. There are test classes defined at the test_datetime module level that subclass from datetime classes. The self.module is not available at the module level. These should probably be moved to setUp() methods and attached to test case self.
  3. If #2 is resolved by moving definitions inside functions, the classes will become unpickleable and pickle tests will break. Some hackery involving injecting these classes into main or module globals may be required.

These challenges make datetime.py an interesting showcase for other modules, so rather than writing a PEP based on abstract ideas, I think it is better to get datetime.py integrated first and try to establish the best practices on the way.

More information about the Python-Dev mailing list