[Python-Dev] Unicode 5.1.0 (original) (raw)
Fredrik Lundh fredrik at pythonware.com
Fri Aug 22 17:13:08 CEST 2008
- Previous message: [Python-Dev] Unicode 5.1.0
- Next message: [Python-Dev] Unicode 5.1.0
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Fri, Aug 22, 2008 at 4:59 PM, Guido van Rossum <guido at python.org> wrote:
(how's the 3.2/4.1 dual support implemented? do we have two distinct datasets, or are the differences encoded in some clever way? would it make sense to split the unicodedata module into three separate modules, one for each major Unicode version?) The current API looks fine to me: unicodedata is the latest version whereas unicodedata.ucd320 is the older version. The APIs are the same; there's a tiny bit of code in the generated db.h file that expresses the differences: static const changerecord* getchange320(PyUCS4 n) { int index; if (n >= 0x110000) index = 0; else { index = changes320index[n>>7]; index = changes320data[(index<<7)+(n & 127)]; } return changerecords320+index; }
there's a bunch of data tables as well, but they don't seem to be very large. looks like Martin did a thorough job here.
... digging digging digging ...
yes, the generator script produces difference tables between the main version and a list of older versions. I'd say it's worth running the script on the 5.1.0 tables, and if it doesn't choke, compare the resulting table with the corresponding table for 4.1.0 (a simple loop fetching the main properties for all code points). if the differences look reasonably small, switch 5.1.0 and keep the others.
I can tinker a little with this over the weekend, unless Martin tells me not to ;-)
- Previous message: [Python-Dev] Unicode 5.1.0
- Next message: [Python-Dev] Unicode 5.1.0
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]