tft.word_count | TFX | TensorFlow (original) (raw)
tft.word_count
Stay organized with collections Save and categorize content based on your preferences.
Find the token count of each document/row.
tft.word_count(
tokens: Union[tf.SparseTensor, tf.RaggedTensor], name: Optional[str] = None
) -> tf.Tensor
tokens is either a RaggedTensor or SparseTensor, representing tokenized strings. This function simply returns size of each row, so the dtype is not constrained to string.
Example:
sparse = tf.SparseTensor(indices=[[0, 0], [0, 1], [2, 2]],
values=['a', 'b', 'c'], dense_shape=(4, 4))
tft.word_count(sparse)
<tf.Tensor: shape=(4,), dtype=int64, numpy=array([2, 0, 1, 0])>
| Args | |
|---|---|
| tokens | either (1) a SparseTensor, or (2) a RaggedTensor with ragged rank of 1, non-ragged rank of 1 of dtype tf.string containing tokens to be counted |
| name | (Optional) A name for this operation. |
| Returns |
|---|
| A one-dimensional Tensor the token counts of each row. |
| Raises | |
|---|---|
| ValueError | if tokens is neither sparse nor ragged |
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-11-01 UTC.