Issue 653938: Adds a builtin decimal type (FixedPoint) (original) (raw)

This patch integrates the fixedpoint module, that was created by Tim Peters, into Python as a first class object. That is, the patch adds this new number type so that it has direct syntax support like float, int, long, str., etc. within the interpreter.

I use Tim's module to implement the type. This limits the patch to a small bit of code that adds the a syntax interface to this module. Syntax recognition for the new number format required a change to Parser/tokenizer.c and to Python/compile.c. This patch allows the new decimal type (I renamed the FixedPoint type to decimal in the fixedpoint.py file because the name is shorter and it is sufficient to distinguish the type from a binary float.) to be entered as a literal that is similar to a long, int, or float. The new syntax works as follows:

12.00d 12.00d .012d 0.012d 1d 1.d str(1.003d) '1.003'

As you can see from the example, the default precision for decimal literals are determined by the precision entered expressed in the literal.

The patch also adds glue to Python/bltinmodule.c to create a builtin decimal() function. This builtin decimal (Formally FixedPoint) function invokes the constructor of the Python implementation of the decimal class defined in the fixedpoint module. The implementation follows the familiar pattern for adding special builtin functions, much like the implementation of apply and abs. It does not follow the typical method for adding built in types. That would require a more invasive patch. The builtin decimal function works as follows:

decimal(1.333) 1.33d decimal(1.333,4) 1.3330d decimal(400,2) 400.00d

The semantics of the precision parameter may be incorrect because it is only addressing the precision of the fraction part of the number. Shouldn't it reflect the number of significant digits? If that were the case then the following would have been the result:

decimal(400,2) 4.0e2d

This problem is more noticable when exponents are large relative to the number of significant digits. For instance:

40.e3d 40000.d

The result should have been 40.e3d, not 40000. This implies more precision than is declared by the constant.

As it currently is implemented the type pads with zeros on the integer side of the decimal point, whic implies more accuracy in the number than is true. I ran into this problem when I tried to implement the automatic conversion of string number values to decimal representation for numbers in 'e' format.

3.03e5d 303000.00d

The representation is incorrect. It should have returned 3.03e5d but it padded the result with bogus zeros.

For this first cut at a new decimal type I am primarily interested in investigating the semantics of the new native decimal number type in Python. This type will be useful to bankers, accountants, and newbies. The backend implementation could be replaced with a C implementation without affecting the Python language semantics.

The approach used in this patch makes it very easy to experiment with changes to the semantics because all of the experimentation can be done by changing the code in Lib/fixedpoint.py. The compiled wrapper doesn't have to be modified and recompiled.

Unit testing so far just reuses the _test function in the fixedpoint.py module. Since I am not sure which way to go with the semantic interpretation of precision I decided to post the patch before making a significant change to fixedpoint.py. Some feedback on the interpretation of precision would be appreciated.

Documentation:

The following needs to be added to section 2.1 of the library reference manual:

decimal(value, precision) Convert a string or a number to fixed point decimal number. If the argument is a string, it must contain a possibly signed decimal or floating point number, possibly embedded in whitespace. The precision parameter will be ignored on string values. The precision will be set based on the number of significant digits. Otherwise, the argument may be a plain or long integer or a floating point number, and a decimal number with the same value ( the precision attribute will set the precision of the fraction roundoff) is returned.

Section 3.1.1 of the tutorial will add a brief description of decimal number usage after the description of floating point:

There is full support for floating point; operators with mixed type operands convert the integer operand to floating point:

3 * 3.75 / 1.5 7.5 7.0 / 2 3.5

Python also supports fixed point decimal numbers. These numbers do not suffer from the somewhat random roundoff errors that can occur with binary floating point numbers. A decimal number is created by adding a 'd' or 'D' suffix to a number:

3.3d + 3d 6.3d 3.3d + 3.03 6.3d 3.3d + decimal(3.03,3) 6.330d decimal(1.1, 16) 1.1000000000000001d

The builtin decimal constructor provides a means for converting float and int types to decimal types. Since floats are approximations of decimal floating point numbers there are often roundoff errors introduced by using floats. (The 1 in the last place of the conversion of 1.1 with 16 digits is a binary round off error.) For this reason the decimal function requires the specification of precision when converting a float to a decimal. This allows the significant digits of the number to be specified. (Accountants and bankers love this because it allows them to balance the books without pennies being lost due to the use of binary numbers.)

3.3d/2 1.6d

Note that in the example above the expression 3.3d/2 returned 1.6d. The rounding scheme for Python decimals uses "banking" rounding rules. With floating point numbers the result would have been as follows:

3.3/2 1.6499999999999999

So as you can see the banking rules round off the .04999999999 portion of the number and calls it a an even 1.6.