[Python-Dev] csv module TODO list (original) (raw)

Anders J. Munch ajm at flonidan.dk
Thu Jan 6 17:22:12 CET 2005

Andrew McNamara wrote:

I'm not altogether sure there. The parsing state machine is all written in C, and deals with signed chars - I expect we'll need two versions of that (or one version that's compiled twice using pre-processor macros). Quite a large job. Suggestions gratefully received.

How about using UTF-8 internally? Change nothing in _csv.c, but in csv.py encode/decode any unicode strings into UTF-8 on the way to/from _csv. File-like objects passed in by the user can be wrapped in proxies that take care of encoding and decoding user strings, as well as trans-coding between UTF-8 and the users chosen file encoding.

All that coding work may slow things down, but your original fast _csv module will still be there when you need it.

More information about the Python-Dev mailing list