spmd - Execute code in parallel on workers of parallel pool - MATLAB (original) (raw)

Execute code in parallel on workers of parallel pool

Syntax

Description

spmd, _`statements`_, end defines an spmd statement on a single line. MATLAB® executes the spmd body denoted bystatements on several MATLAB workers simultaneously. Each worker can operate on a different data set or different portion of distributed data, and can communicate with other participating workers while performing the parallel computations. Thespmd statement can be used only if you have Parallel Computing Toolbox™. To execute the statements in parallel, you must first create a pool of MATLAB workers using parpool or have your parallel settings allow the automatic start of a pool.

Inside the body of the spmd statement, each MATLAB worker has a unique value of spmdIndex, while spmdSize denotes the total number of workers executing the block in parallel. Within the body of the spmd statement, communication functions for communicating jobs (such as spmdSend and spmdReceive) can transfer data between the workers.

Values returning from the body of an spmd statement are converted toComposite objects on the MATLAB client. A Composite object contains references to the values stored on the remote MATLAB workers, and those values can be retrieved using cell-array indexing. The actual data on the workers remains available on the workers for subsequentspmd execution, so long as the Composite exists on the client and the parallel pool remains open.

By default, MATLAB uses all workers in the pool. When there is no pool active, MATLAB will create a pool and use all the workers from that pool. If your settings do not allow automatic pool creation, MATLAB executes the block body locally and creates Composite objects as necessary. You cannot execute an spmd block if any worker is busy executing a parfeval request, unless you usespmd(0).

For more information about spmd and Composite objects, see Distribute Arrays and Run SPMD.

example

spmd(_`n`_), _`statements`_, end uses n to specify the exact number of MATLAB workers to evaluate statements, provided that n workers are available from the parallel pool. If there are not enough workers available, an error is thrown. If n is zero, MATLAB executes the block body locally and creates Composite objects, the same as if there is no pool available.

example

spmd(_`m`_,_`n`_), statements, end uses a minimum of m and a maximum of n workers to evaluate statements. If there are not enough workers available, an error is thrown. m can be zero, which allows the block to run locally if no workers are available.

example

spmd([pool](#mw%5F7bbe7287-ae90-4964-9183-65d4ca8091ed),___), statements, end evaluates statements on the parallel pool specified by theparallel.Pool objectpool. Use this syntax when you want to evaluatespmd statements on a pool other than the pool the gcp function returns. (since R2025a)

example

Examples

collapse all

Create a parallel pool, and perform a simple calculation in parallel using spmd. MATLAB executes the code inside the spmd on all workers in the parallel pool.

Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 3).

spmd q = magic(spmdIndex + 2); end

Plot the results.

figure subplot(1,3,1), imagesc(q{1}); subplot(1,3,2), imagesc(q{2}); subplot(1,3,3), imagesc(q{3});

When you are done with computations, you can delete the current parallel pool.

If you have access to several GPUs, you can perform your calculations on multiple GPUs in parallel using a parallel pool.

To determine the number of GPUs that are available for use in MATLAB, use the gpuDeviceCount function.

availableGPUs = gpuDeviceCount("available")

Start a parallel pool with as many workers as available GPUs. For best performance, MATLAB assigns a different GPU to each worker by default.

parpool("Processes",availableGPUs);

Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 3).

To identify which GPU each worker is using, call gpuDevice inside an spmd block. The spmd block runs gpuDevice on every worker.

Use parallel language features, such as parfor or parfeval, to distribute your computations to workers in the parallel pool. If you use gpuArray enabled functions in your computations, these functions run on the GPU of the worker. For more information, see Run MATLAB Functions on a GPU. For an example, see Run MATLAB Functions on Multiple GPUs.

When you are done with your computations, shut down the parallel pool. You can use the gcp function to obtain the current parallel pool.

If you want to use a different choice of GPUs, then you can use gpuDevice to select a particular GPU on each worker, using the GPU device index. You can obtain the index of each GPU device in your system using the gpuDeviceCount function.

Suppose you have three GPUs available in your system, but you want to use only two for a computation. Obtain the indices of the devices.

[availableGPUs,gpuIndx] = gpuDeviceCount("available")

Define the indices of the devices you want to use.

Start your parallel pool. Use an spmd block and gpuDevice to associate each worker with one of the GPUs you want to use, using the device index. The spmdIndex function identifies the index of each worker.

parpool("Processes",numel(useGPUs));

Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 2).

spmd gpuDevice(useGPUs(spmdIndex)); end

As a best practice, and for best performance, assign a different GPU to each worker.

When you are done with your computations, shut down the parallel pool.

Since R2025a

Start a parallel pool using the remote cluster profile myMJSCluster.

myClusterPool = parpool("myMJSCluster",15);

Starting parallel pool (parpool) using the 'myMJSCluster' profile ... Connected to parallel pool with 15 workers.

Partition a pool of 6 workers from the myClusterPool pool.

spmdWorkers = myClusterPool.Workers(1:6); [spmdPool,otherPool] = partition(myClusterPool,"Workers",spmdWorkers);

Calculate an estimate of pi using the workers of the spmdPool pool.

fun = @(x) 4./(1 + x.^2); spmd(spmdPool) a = (spmdIndex - 1)/spmdSize; b = spmdIndex/spmdSize; myIntegral = integral(fun,a,b); piApprox = spmdPlus(myIntegral); end

approx1 = piApprox{1}; fprintf("pi : %.18f\n" + ... "Approximation: %.18f\n" + ... "Error : %g\n", pi,approx1,abs(pi - approx1));

pi : 3.141592653589793116 Approximation: 3.141592653589793116 Error : 0

Input Arguments

collapse all

Since R2025a

Pool evaluating spmd statements, specified as aparallel.Pool object.

Tips

Extended Capabilities

Version History

Introduced in R2008b

expand all

You can now specify the parallel pool that the spmd function uses to evaluate statements. Use this syntax to evaluatespmd statements on a pool other than the pool thegcp function returns.