matlab.io.datastore.MiniBatchable.read - (Not recommended) Read data from custom mini-batch datastore - MATLAB (original) (raw)

• Documentation
• Examples
• Functions
• Blocks
• Apps
• Videos
• Answers

Main Content

Class: matlab.io.datastore.MiniBatchable
Namespace: matlab.io.datastore

(Not recommended) Read data from custom mini-batch datastore

Syntax

Description

[data](#d126e347251) = read(`ds`) returns data from a mini-batch datastore. Subsequent calls to theread function continue reading from the endpoint of the previous call.

[[data](#d126e347251),[info](#d126e347285)] = read(`ds`) also returns information about the extracted data in info, including metadata.

Output Arguments

expand all

Output data, returned as a table with [MiniBatchSize](matlab.io.datastore.minibatchable-class.html#mw%5Fab63bc83-db9b-4105-aa14-dfe18234ca5e) number of rows. For the last mini-batch of data in the datastore, if [NumObservations](matlab.io.datastore.minibatchable-class.html#mw%5F0d4a9770-9765-4f00-a40e-39f577fbe5d0) is not evenly divisible byMiniBatchSize, then data should contain the remaining observations in the datastore (a partial batch smaller than MiniBatchSize).

The table should have two columns, with predictors in the first column and responses in the second column.

Information about read data, returned as a structure array.

Version History

Introduced in R2018a

collapse all

Before R2018a, to perform custom image preprocessing for training deep learning networks, you had to specify a custom read function using thereadFcn property of imageDatastore. However, reading files using a custom read function was slow becauseimageDatastore did not prefetch files.

In R2018a, four classes includingmatlab.io.datastore.MiniBatchable were introduced as a solution to perform custom image preprocessing with support for prefetching, shuffling, and parallel training. Implementing a custom mini-batch datastore usingmatlab.io.datastore.MiniBatchable has several challenges and limitations.

• In addition to specifying the preprocessing operations, you must also define properties and methods to support reading data in batches, reading data by index, and partitioning and shuffling data.
• You must specify a value for the NumObservations property, but this value may be ill-defined or difficult to define in real-world applications.
• Custom mini-batch datastores are not flexible enough to support common deep learning workflows, such as deployed workflows using GPU Coder™.

Starting in R2019a, built-in datastores natively support prefetch, shuffling, and parallel training when reading batches of data. The transform function is the preferred way to perform custom data preprocessing, or transformations. The combine function is the preferred way to concatenate read data from multiple datastores, including transformed datastores. Concatenated data can serve as the network inputs and expected responses for training deep learning networks. The transform and combine functions have several advantages over matlab.io.datastore.MiniBatchable.

• The functions enable data preprocessing and concatenation for all types of datastores, including imageDatastore.
• The transform function only requires you to define the data processing pipeline.
• When used on a deterministic datastore, the functions supporttall data types and MapReduce.
• The functions support deployed workflows.

There are no plans to remove the read method ofmatlab.io.datastore.MiniBatchable at this time.