| msg61148 - (view) |
Author: Fredrik Johansson (fredrik_j) |
Date: 2004-01-29 20:34 |
| I think it'd be useful with a built-in module (I propose the name 'mint' :), providing classes for emulating machine ints. There should be support for ints of different sizes (at least 8, 16, 32, 64), signedness and endianness. The machine ints would react to overflow (negative values) and bit-level operations in the same way their C equivalents would. One idea for a feature would be a intobj.pack() (or str() or whatever) method that could be used instead of the struct module's pack(). int16(12345).pack() -> "\x30\x39". int32(-1).pack() -> "\0xff\0xff\0xff\0xff", etc. A related idea is that these ints could provide slicing to extract individual sets of bits or bytes. For example, uint32()[8:16] could return an int (uint8?) made from the bits in the int's second byte (whether it's the second from the left or the right might be fuzzy, I do realize). Applications for this would be algorithmic code involving bit fiddling (useful for cryptographic applications, perhaps) and, obviously, programs that have to operate on any binary data. It might also resolve some issues related to the unification of longs and ints. |
|
|
| msg61149 - (view) |
Author: Raymond Hettinger (rhettinger) *  |
Date: 2004-01-30 00:35 |
| Logged In: YES user_id=80475 FTI, there was a related feature request, www.python.org/sf/846568 for treating numbers as bitarrays. As proposed, it was invalid because numbers in Python are immutable (no bit assignments allowed). That could be combined with this idea generic numeric register class that optionally specifies size (if bound), base, endianess, etc. bcdreg = register(places=5, base=10) bcdreg.assign(697) bcdreg[-1] --> 7 bcdreg.multiply(1000) --> 97000 None of this is for the mainstream user and it would be unlikely to ever be included in the standard library. The best approach would be to experiment with an API in pure python, perhaps publish it as a recipe, and if it is well received, put a C version on the vaults of parnassus. |
|
|
| msg61150 - (view) |
Author: Fredrik Johansson (fredrik_j) |
Date: 2004-01-30 14:08 |
| Logged In: YES user_id=756835 As for mainstream uses, I think replacing struct.pack would be one. |
|
|
| msg61151 - (view) |
Author: Thomas Heller (theller) * |
Date: 2004-01-30 14:16 |
| Logged In: YES user_id=11105 You might look at the ctypes module (google for it) - it implements mutable C compatible data types. |
|
|
| msg61152 - (view) |
Author: Bob Ippolito (bob.ippolito) * |
Date: 2004-02-26 02:43 |
| Logged In: YES user_id=139309 If I were providing slicing on bits I would say that foo[n] is bool(foo & (1<<n)) .. which means bit 0 is least significant, and bit N is most significant. I don't think any other way would make sense. |
|
|
| msg61153 - (view) |
Author: Armin Rigo (arigo) * |
Date: 2004-05-21 16:01 |
| Logged In: YES user_id=4771 A number of projects end up redefining Python classes that try to emulate the raw integers of C, which is a pain to do properly in Python. I think that a general-purpose 'mint' class would be generally useful. Writing them in Python is a bit of a pain (http://codespeak.net/svn/pypy/trunk/src/pypy/objspace/std/restricted_int.py). Pyrex seems to be a good way to generate it as a extension module (http://codespeak.net/svn/user/arigo/misc/machineint.pyx). |
|
|
| msg75815 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2008-11-13 10:45 |
| Is it be feasible to add arithmetic operations to the ctypes integer types? Since ctypes is now in the core, it would seem better to enhance ctypes than provide a new module. I think this would be valuable for rapid prototyping of an algorithm in Python before translating it into C. In particular, it might help with detecting some common C code bugs involving undefined behaviour---for example, overflow in signed-integer arithmetic. |
|
|
| msg75916 - (view) |
Author: Thomas Heller (theller) * |
Date: 2008-11-15 18:28 |
| I wonder if a patch for ctypes like this (which is not yet complete) could be used to implement this, or MUST it be implemented in C? The patch contains a mixin class that implements the numeric methods. However, the actual operation takes place in Python; only the operands are converted into ctypes types first, the operand is applied to the value, and the result is converted to a ctypes instance again. One difficulty with the patch is that the original ctypes code contained a tp_as_number member where only the nb_nonzero slot was actually implemented; this prevented the mixin class to do it's work (Python didn't call the special methods. I wonder if there is a bug somewhere...). |
|
|
| msg77570 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2008-12-10 21:28 |
| > I wonder if a patch for ctypes like this (which is not yet complete) > could be used to implement this, or MUST it be implemented in C? For the intended purpose of giving access to raw machine arithmetics, I would think a C implementation would be required, but Theller's patch is a good start. > One difficulty with the patch is that the original ctypes code > contained a tp_as_number ... This can be solved by changing the order of bases for c_ classes. See attached. The next step would be to rewrite _NumberMixin in C. C code can be limited to methods operating on known c_ types, say ll_add(x, y) to add two longs and hl_cast(x) to cast short to long with python code taking care of the dispatch. (What's the latest on the multiple dispatch, BTW? ) |
|
|
| msg77571 - (view) |
Author: Thomas Heller (theller) * |
Date: 2008-12-10 21:41 |
| >> One difficulty with the patch is that the original ctypes code >> contained a tp_as_number ... >This can be solved by changing the order of bases for c_ classes. >See attached. Cool! Why didn't I think of that myself? |
|
|
| msg100426 - (view) |
Author: Alexander Belopolsky (Alexander.Belopolsky) |
Date: 2010-03-04 22:20 |
| Attaching a patch that implements machine arithmetics for numeric ctypes in C. |
|
|
| msg100442 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2010-03-05 00:06 |
| I don't care much for ctypes but I think the amount of code duplication in that patch is scary. |
|
|
| msg100448 - (view) |
Author: Alexander Belopolsky (Alexander.Belopolsky) |
Date: 2010-03-05 02:21 |
| Code duplication is unavoidable because the goal is to give access to machine arithmetics which means (# types) x (# operations) of very similar looking functions. I considered reducing (perceived) code duplication through some pre-processor magic, but doing so would make code much harder to understand and almost impossible to debug. On Thu, Mar 4, 2010 at 7:06 PM, Antoine Pitrou <report@bugs.python.org> wrote: > > Antoine Pitrou <pitrou@free.fr> added the comment: > > I don't care much for ctypes but I think the amount of code duplication in that patch is scary. > > ---------- > nosy: +pitrou > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue887237> > _______________________________________ > |
|
|
| msg100449 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2010-03-05 02:26 |
| Le Fri, 05 Mar 2010 02:21:30 +0000, Alexander Belopolsky <report@bugs.python.org> a écrit : > > Code duplication is unavoidable because the goal is to give access to > machine arithmetics which means (# types) x (# operations) of very > similar looking functions. I considered reducing (perceived) code > duplication through some pre-processor magic, but doing so would make > code much harder to understand and almost impossible to debug. Carefully written macros shouldn't be hard to understand. At least there wouldn't be the risk of overlooking one of the methods when making modifications. |
|
|
| msg100450 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2010-03-05 02:29 |
| Look at Modules/operator.c for example of an effective use of macros to minimize code duplication. |
|
|
| msg100451 - (view) |
Author: Alexander Belopolsky (Alexander.Belopolsky) |
Date: 2010-03-05 02:46 |
| I would like to hear from Thomas before introducing macros in this code. I tried to follow the style of cfield.c which shows similar code duplication. There are also some questions that need to be answered before polishing the code. 1. Should mixed arithmetics be supported? 2. Should we do anything special about floating point operations? Wrapping them in PyFPE_START/STOP_PROTECT? 3. Should we support in-place operations? 4. Bitwise operations on integers? On Thu, Mar 4, 2010 at 9:29 PM, Raymond Hettinger <report@bugs.python.org> wrote: > > Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment: > > Look at Modules/operator.c for example of an effective use of macros to minimize code duplication. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue887237> > _______________________________________ > |
|
|
| msg100496 - (view) |
Author: Thomas Heller (theller) * |
Date: 2010-03-05 19:17 |
| >> Code duplication is unavoidable because the goal is to give access to >> machine arithmetics which means (# types) x (# operations) of very >> similar looking functions. I considered reducing (perceived) code >> duplication through some pre-processor magic, but doing so would make >> code much harder to understand and almost impossible to debug. > > Carefully written macros shouldn't be hard to understand. > At least there wouldn't be the risk of overlooking one of the methods > when making modifications. The code duplication in Alex's patch is indeed enormous. I can see two solutions to avoid (or work around) them. Use of macros, or writing a Python script that generates the C-code. Anyway, I guess his patch is some example code, not more. (Using C++ overloaded functions or even templates would be another - but only theoretical - possibility). > I would like to hear from Thomas before introducing macros in this > code. I tried to follow the style of cfield.c which shows similar > code duplication. > > There are also some questions that need to be answered before > polishing the code. > > 1. Should mixed arithmetics be supported? > 2. Should we do anything special about floating point operations? > Wrapping them in PyFPE_START/STOP_PROTECT? > 3. Should we support in-place operations? > 4. Bitwise operations on integers? I would answer 1, 3, and 4, with 'yes'. However, this grows the code size by another dimension since we now have #types * #operation * #types. My answer for question 3 is 'I don't know'. Before I forget: It may be possible to implement ctypes number methods as a third-party module by implementing a mixin class, and replacing the c_... type definitions in Lib/ctypes/__init__.py. |
|
|
| msg100512 - (view) |
Author: Alexander Belopolsky (Alexander.Belopolsky) |
Date: 2010-03-06 02:50 |
| Attaching a patch that is equivalent toissue887237.diff, but uses preprocessor to generate repetitive code. |
|
|
| msg110339 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2010-07-14 23:09 |
| Perhaps people could review the latest patch in view of the following. 1) Quote from Mark Dickinson "I think this would be valuable for rapid prototyping of an algorithm in Python before translating it into C. In particular, it might help with detecting some common C code bugs involving undefined behaviour---for example, overflow in signed-integer arithmetic." 2) "Do not assume signed integer overflow behavior" Perhaps this could be treated as purely experimental? (This might have been suggested already, sorry if I've missed it). Whatever happens I'd be glad to help out if possible, the big asset I have compared to others is time as I only work part time. |
|
|
| msg123002 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2010-12-01 18:32 |
| As far as I understand, the main concern about .diff was code duplication. There are two ways to fight it: C preprocessor tricks as in -macro.diff and code generation as done in numpy. With improved macro support in many compilers and debuggers, preprocessor approach may be preferable. However I cannot move this issue forward until those who objected to code duplication review the latest patch. Unassigning. |
|
|
| msg132277 - (view) |
Author: Benjamin Peterson (benjamin.peterson) * |
Date: 2011-03-26 22:13 |
| You could write a Python script to generator the methods. |
|
|