Normalization Corrigendum (original) (raw)

In the production of the normalization tables for Unicode 3.0, the character U+FB1D HEBREW LETTER YOD WITH HIRIQ was mistakenly omitted from Composition Exclusions. During the public review period, this mistake was reported, but the report was misinterpreted and thus overlooked. In Unicode 3.1, this character is now included in Composition Exclusions.

This particular change does affect backwards compatibility of Normalization forms NFKC and NFC for strings containing this character. It is recommended that all implementations for these Normalization forms upgrade to the Unicode 3.1 data tables.

Policies. The Unicode Technical Committee has authorized a change in Composition Exclusions in Unicode 3.1 to correct this omission. The officers of the Unicode consortium have also approved this change. The reasons for the exceptional decision are that:

• The omission had been reported during the public review period for Unicode 3.0.
• There were no normative references to Unicode 3.0 Normalization from our liaison organizations (particularly IETF and W3C), although normative references are expected soon after Unicode 3.1.
• YOD WITH HIRIQ is one of a class of characters ("marked" Hebrew presentation forms within the range U+FB1D .. U+FB4E) that were to be handled all in the same way, during all review and discussion of Normalization in the UTC. The other characters in this class were uniformly included in Composition Exclusions.
• YOD WITH HIRIQ is a very rare character. The amount of existing data containing it is infinitesimal as a proportion of all computerized text. Even if it takes some time for implementations to upgrade, this change should pose no significant backwards-compatibility issue in practice.

No future changes will be made to normalization that would affect backwards compatibility; no other characters could satisfy these criteria in the future.