tf.strings.unicode_encode | TensorFlow v2.16.1 (original) (raw)

tf.strings.unicode_encode

Stay organized with collections Save and categorize content based on your preferences.

Encodes each sequence of Unicode code points in input into a string.

View aliases

Compat aliases for migration

tf.compat.v1.strings.unicode_encode

tf.strings.unicode_encode(
    input,
    output_encoding,
    errors='replace',
    replacement_char=65533,
    name=None
)

Used in the notebooks

Used in the guide
Unicode strings Tokenizing with TF Text

result[i1...iN] is the string formed by concatenating the Unicode codepoints input[1...iN, :], encoded using output_encoding.

Args
input	An N+1 dimensional potentially ragged integer tensor with shape[D1...DN, num_chars].
output_encoding	Unicode encoding that should be used to encode each codepoint sequence. Can be "UTF-8", "UTF-16-BE", or "UTF-32-BE".
errors	Specifies the response when an invalid codepoint is encountered (optional). One of: * `'replace'`: Replace invalid codepoint with the `replacement_char`. (default) * `'ignore'`: Skip invalid codepoints. * `'strict'`: Raise an exception for any invalid codepoint.
replacement_char	The replacement character codepoint to be used in place of any invalid input when errors='replace'. Any valid unicode codepoint may be used. The default value is the default unicode replacement character which is 0xFFFD (U+65533).
name	A name for the operation (optional).

Returns
A N dimensional string tensor with shape [D1...DN].

Example:

input = tf.ragged.constant( [[71, 246, 246, 100, 110, 105, 103, 104, 116], [128522]]) print(unicode_encode(input, 'UTF-8')) tf.Tensor([b'G\xc3\xb6\xc3\xb6dnight' b'\xf0\x9f\x98\x8a'], shape=(2,), dtype=string)