[Python-Dev] PEP 460: allowing %d and %f and mojibake (original) (raw)

Stephen J. Turnbull stephen at xemacs.org
Mon Jan 13 15:43:02 CET 2014

Glenn Linderman writes:

On 1/12/2014 4:08 PM, Stephen J. Turnbull wrote:

Glenn Linderman writes:

the proposals to embed binary in Unicode by abusing Latin-1 encoding.

Those aren't "proposals", they are currently feasible techniques in Python 3 for some use cases. The question is why infecting Python 3 with the byte/character confoundance virus is preferable to such techniques, especially if their (serious!) deficiencies are removed by creating a new type such as asciistr.

"smuggled binary" (great term borrowed from a different subthread) muddies the waters of what you are dealing with.

Not really. The "mud" is one or more of the serious deficiencies. It can be removed, I believe (and Nick apparently does, too). "asciistr" is one way to try that.

When the mixture of text and binary is done as encoded text in binary, then it is obvious that only limited text processing can be performed,

Hardly. After all, that's how all text processing was done for decades. Still is, in some programs, especially C programs.

And there are no extra, confusing Latin-1 encode/decode operations required.

The "extra" encode/decode operations are mostly (perhaps all) due to examples that started from bytes and end with bytes. Of course if you assume that API and propose to do the operations using Unicode, you'll get "extra" decode/encode operations.

From a higher-level perspective, I think it would be great to have a module, perhaps called "boundary" (let's call it that for now), that allow some definition syntax (augmented BNF? augmented ABNF?) to explain the format of a binary blob.

We have struct, for one. I'm not sure why you want more than that. I suppose you could go all the way to ASN.1.

More information about the Python-Dev mailing list