numpartitions - Number of datastore partitions - MATLAB (original) (raw)

• Documentation
• Examples
• Functions
• Apps
• Videos
• Answers

Main Content

Number of datastore partitions

Syntax

Description

n = numpartitions([ds](#buouy30-1-ds)) returns the default number of partitions for datastore ds.

example

n = numpartitions([ds](#buouy30-1-ds),[pool](#buouy30-1-pool)) returns a number of partitions to parallelize datastore access over the parallel pool specified by pool. To parallelize datastore access, you must have Parallel Computing Toolbox™ installed.

example

Examples

collapse all

Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.

ds = datastore('mapredout.mat');

Get the default number of partitions.

By default, there is only one partition in ds because it contains only one small file.

Partition the datastore and return the datastore corresponding to the first part.

subds = partition(ds,n,1);

Read the data in subds.

while hasdata(subds) data = read(subds); end

Get a number of partitions to parallelize datastore access over the current parallel pool. You must have Parallel Computing Toolbox installed.

Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.

ds = datastore('mapredout.mat');

Get a number of partitions to parallelize datastore access over the current parallel pool.

n = numpartitions(ds, gcp);

Partition the datastore and read the data in each part.

parfor ii=1:n subds = partition(ds,n,ii); while hasdata(subds) data = read(subds); end end

Input Arguments

collapse all

Input datastore. You can use the datastore function to create a datastore object from your data.

Parallel pool object.

Example: gcp

Extended Capabilities

expand all

Usage notes and limitations:

• In a thread-based environment, you can use numpartitions only with the following datastores:
  • ImageDatastore objects
  • CombinedDatastore, SequentialDatastore, orTransformedDatastore objects you create fromImageDatastore objects by usingcombine or transform
    You can use numpartitions with other datastores if you have Parallel Computing Toolbox. To do so, run the function using a process-backed parallel pool instead of using backgroundPool or ThreadPool (use eitherProcessPool or ClusterPool).

For more information, see Run MATLAB Functions in Thread-Based Environment.

Version History

Introduced in R2015a