partition - Partition minibatchqueue - MATLAB (original) (raw)

Main Content

Partition minibatchqueue

Since R2020b

Syntax

Description

[submbq](#mw%5Fc7da613b-7ba4-440d-88fa-84524e74e2ff) = partition([mbq](#mw%5F3d7ade8d-0d0f-490b-a1fb-fac0c10a5cab%5Fsep%5Fmw%5F1052debc-abad-4547-9b2c-423b954fdb8a),[numParts](#mw%5F51ecd613-7adf-462a-8852-df8315464e02),[indx](#mw%5F6791060f-bb70-4e2b-95c0-1a847ef5202a)) partitions the minibatchqueue objectmbq into numParts parts and returns the partition corresponding to the index indx. The properties ofsubmbq are the same as the properties of mbq.

The output minibatchqueue object has access only to the partition of data it is given when it is created. Using reset withsubmbq resets the minibatchqueue object to the start of the data partition. Using shuffle withsubmbq shuffles only the partitioned data. If you want to shuffle the data across multiple partitions, you must shuffle the originalminibatchqueue object and then re-partition.

example

Examples

collapse all

Use the partition function to divide aminibatchqueue object into three parts.

Create a minibatchqueue object from a datastore.

ds = digitDatastore; mbq = minibatchqueue(ds)

mbq = minibatchqueue with 1 output and properties:

Mini-batch creation: MiniBatchSize: 128 PartialMiniBatch: 'return' MiniBatchFcn: 'collate' PreprocessingEnvironment: 'serial'

Outputs: OutputCast: {'single'} OutputAsDlarray: 1 MiniBatchFormat: {''} OutputEnvironment: {'auto'}

Partition the minibatchqueue object into three parts and return the first partition.

sub1 = partition(mbq,3,1)

sub1 = minibatchqueue with 1 output and properties:

Mini-batch creation: MiniBatchSize: 128 PartialMiniBatch: 'return' MiniBatchFcn: 'collate' PreprocessingEnvironment: 'serial'

Outputs: OutputCast: {'single'} OutputAsDlarray: 1 MiniBatchFormat: {''} OutputEnvironment: {'auto'}

sub1 contains approximately the first third of the data inmbq.

Use the partition function to divide aminibatchqueue object into three parts.

Create a minibatchqueue object from a datastore.

ds = digitDatastore; mbq = minibatchqueue(ds)

mbq = minibatchqueue with 1 output and properties:

Mini-batch creation: MiniBatchSize: 128 PartialMiniBatch: 'return' MiniBatchFcn: 'collate' PreprocessingEnvironment: 'serial'

Outputs: OutputCast: {'single'} OutputAsDlarray: 1 MiniBatchFormat: {''} OutputEnvironment: {'auto'}

Partition the minibatchqueue object into three parts on three workers in a parallel pool. Iterate over the data on each worker.

numWorkers = 3; p = parpool("Processes",numWorkers); parfor i=1:3 submbq = partition(mbq,3,i); while hasdata(submbq) data = next(submbq); end end

Each worker has access to a subset of the data in the originalminibatchqueue object.

Input Arguments

collapse all

Number of partitions, specified as a numeric scalar.

Partition index, specified as a numeric scalar.

Output Arguments

collapse all

Output minibatchqueue, specified as aminibatchqueue object. submbq contains a subset of the data in mbq. The properties of submbq are the same as the properties of mbq.

Version History

Introduced in R2020b