tfds.builder_from_directories | TensorFlow Datasets (original) (raw)

tfds.builder_from_directories

Loads a tfds.core.DatasetBuilder from the given generated dataset path.

tfds.builder_from_directories(
    builder_dirs: List[tfds.typing.PathLike],
    *,
    filetype_suffix: Optional[str] = None
) -> tfds.core.DatasetBuilder

When a dataset is spread out over multiple folders, then this function can be used to easily read from all builder dirs.

Note that the data in each folder must have the same features, dataset name, and version.

Some examples of when a dataset might be spread out over multiple folders:

in reinforcement learning, multiple agents each produce a dataset
each day a new dataset is produced based on new incoming events

Arguments
builder_dirs	the list of builder dirs from which the data should be read.
filetype_suffix	DEPRECATED PLEASE DO NOT USE. The filetype suffix (e.g. 'tfrecord') that is used if the file format is not specified in the DatasetInfo.

Returns
the read only dataset builder that is configured to read from all the given builder dirs.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2024-04-26 UTC.