What Is Parallel Computing? - MATLAB & Simulink (original) (raw)
Parallel computing allows you to carry out many calculations simultaneously. Large problems can often be split into smaller ones, which are then solved at the same time.
The main reasons to consider parallel computing are to
- Save time by distributing tasks and executing these simultaneously
- Solve big data problems by distributing data
- Take advantage of your desktop computer resources and scale up to clusters and cloud computing
With Parallel Computing Toolbox™, you can
- Accelerate your code using interactive parallel computing tools, such as
parforandparfeval - Scale up your computation using interactive Big Data processing tools, such as
distributed,tall,datastore, andmapreduce - Use
gpuArrayto speed up your calculation on the GPU of your computer - Use
batchto offload your calculation to computer clusters or cloud computing facilities
Here are some useful Parallel Computing concepts:
- Node: standalone computer, containing one or more CPUs / GPUs. Nodes are networked to form a cluster or supercomputer
- Thread: smallest set of instructions that can be managed independently by a scheduler. On a GPU, multiprocessor or multicore system, multiple threads can be executed simultaneously (multi-threading)
- Batch: off-load execution of a functional script to run in the background
- Scalability: increase in parallel speedup with the addition of more resources
What tools do MATLAB® and Parallel Computing Toolbox offer?
- MATLAB workers: MATLAB computational engines that run in the background without a graphical desktop. You use functions in the Parallel Computing Toolbox to automatically divide tasks and assign them to these workers to execute the computations in parallel. You can run local workers to take advantage of all the cores in your multicore desktop computer. You can also scale up to run your workers on a cluster of machines, using the MATLAB Parallel Server™. The MATLAB session you interact with is known as the_MATLAB client_. The client instructs the workers with parallel language functions.
- Parallel pool: a parallel pool of MATLAB workers created using
parpoolor functions with automatic parallel support. By default, parallel language functions automatically create a parallel pool for you when necessary. To learn more, see Run Code on Parallel Pools.
For the default Processes profile, the default number of workers is one per physical CPU core using a single computational thread. This is because even though each physical core can have several virtual cores, the virtual cores share some resources, typically including a shared floating point unit (FPU). Most MATLAB computations use this unit because they are double-precision floating point. Restricting to one worker per physical core ensures that each worker has exclusive access to a floating point unit, which generally optimizes performance of computational code. If your code is not computationally intensive, for example, it is input/output (I/O) intensive, then consider using up to two workers per physical core. Running too many workers on too few resources may impact performance and stability of your machine. - Speed up: Accelerate your code by running on multiple MATLAB workers or GPUs, for example, using parfor, parfeval, or gpuArray.
- Scale up your data: Partition your big data across multiple MATLAB workers, using tall arrays and distributed arrays. To learn more, see Big Data Processing.
- Asynchronous processing: Use parfeval to execute a computing task in the background without waiting for it to complete.
- Scale up to clusters and clouds: If your computing task is too big or too slow for your local computer, you can offload your calculation to a cluster onsite or in the cloud using MATLAB Parallel Server. For more information, see Clusters and Clouds.