Add modern binary encoding APIs · Issue #6811 · whatwg/html (original) (raw)

Many protocols, APIs, and algorithms require that some binary data (byte array) is serialized into a string that represents that binary data losslessly. Common formats for this are for example base64 encoding and hex encoding. Often the reverse - so deserializing the string back into the original data - is required too.

Here are some (common) use cases that require base64 or hex encoding / decoding some binary data:

• Encoding a png image into a data URL (base64 encoding the png)
• Creating a hex string from a cryptographic digest (hash)
• Generating a random ID from crypto.getRandomValues (hex encoding a random byte array)
• Send binary data over transports that only supports string values (base64 {de/en}coding)
• Parsing PEM files (binary data is stored as base64 encoded strings)

The web platform does not provide a fast an easy approach to base64 / hex encode and decode. Because of this web developers have often resorted to slow and inefficient encoders built on atob/btoa, Number#toString, and parseInt.

I propose to add a new modern binary encoding API as explained in the proposal-binary-encoding repository. This would add a new BinaryEncoder and BinaryDecoder interface (akin to TextEncoder and TextDecoder). The proposed API is simple enough to fit comfortably into section 8.3, "Base64 utility methods" of this spec. The section should likely be renamed to Binary encoding in case this makes it.

For more background reading with usage examples and previous art, a preliminary spec text, and a polyfill please view the above linked repository.

Previous discussions in this repository related to binary encoders: #351 and #6779, specifically this comment: #351 (comment).

A TC39 proposal for a similar issue with lesser scope was opened a few days ago too: https://github.com/bakkot/proposal-arraybuffer-base64. For discussion on the question of "language vs platform feature" and scope, view this issue: tc39/proposal-arraybuffer-base64#4.