[Python-Dev] New PEP: Using ssize_t as the index type (original) (raw)

Travis E. Oliphant oliphant.travis at ieee.org
Fri Dec 30 10:13:00 CET 2005

Martin v. Löwis wrote:

Please let me know what you think.

Regards, Martin PEP: XXX Title: Using ssizet as the index type Version: RevisionRevisionRevision Last-Modified: DateDateDate Author: Martin v. Löwis <martin at v.loewis.de> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 18-Dec-2005 Post-History:

Abstract ======== In Python 2.4, indices of sequences are restricted to the C type int. On 64-bit machines, sequences therefore cannot use the full address space, and are restricted to 2**31 elements. This PEP proposes to change this, introducing a platform-specific index type Pyssizet. An implementation of the proposed change is in http://svn.python.org/projects/python/branches/ssizet.

Sounds wonderful. Would love to see this in Python 2.5. This will fix important 64-bit issues. Perhaps someone could explain to me what the difference between Py_ssize_t and Py_intptr_t would be? Is this not a satisfactory Py_ssize_t already?

Rationale ========= 64-bit machines are becoming more popular, and the size of main memory increases beyond 4GiB. On such machines, Python currently is limited, in that sequences (strings, unicode objects, tuples, lists, array.arrays, ...) cannot contain more than 2GiElements. Today, very few machines have memory to represent larger lists: as each pointer is 8B (in a 64-bit machine), one needs 16GiB to just hold the pointers of such a list; with data in the list, the memory consumption grows even more. However, there are three container types for which users request improvements today: * strings (currently restricted to 2GiB) * mmap objects (likewise; plus the system typically won't keep the whole object in memory concurrently) * Numarray objects (from Numerical Python)

scipy_core objects are the replacement for both Numarray and Numerical Python and support 64-bit clean objects except for the sequence protocol and the buffer protocol. Fixing this problem will clean up a lot of unnecessary code.

Looking forward to it...

-Travis

More information about the Python-Dev mailing list