parallel.Cluster - Access cluster properties and behaviors - MATLAB (original) (raw)
Access cluster properties and behaviors
Container Hierarchy
| Parent | None |
|---|---|
| Children | parallel.Job, parallel.Pool |
Description
A parallel.Cluster object provides access to a cluster, which controls the job queue, and distributes tasks to workers for execution.
Types
The two categories of clusters are the MATLAB Job Scheduler and common job scheduler (CJS). The MATLAB Job Scheduler is available in the MATLAB Parallel Server™. The CJS clusters encompass all other types, including the local, generic, and third-party schedulers.
Use MJSComputeCloud objects to interact with MATLAB Parallel Server for Amazon EC2® clusters.
The following table describes the available types of cluster objects.
| Cluster Type | Description |
|---|---|
| parallel.cluster.MJS | Interact with MATLAB Job Scheduler clusters on-premises |
| parallel.cluster.MJSComputeCloud | Interact with MATLAB Parallel Server for clusters in the cloud |
| parallel.cluster.Local | Interact with CJS cluster running locally on client machine |
| parallel.cluster.HPCServer | Interact with CJS cluster running Windows® Microsoft® HPC Server |
| parallel.cluster.LSF | Interact with CJS cluster running LSF® |
| parallel.cluster.PBSPro | Interact with CJS cluster running Altair PBS Pro® or OpenPBS |
| parallel.cluster.Torque | Interact with CJS cluster running TORQUE |
| parallel.cluster.Slurm | Interact with CJS cluster running Slurm |
| parallel.cluster.Generic | Interact with CJS cluster using the generic interface |
Methods
MATLAB Job Scheduler
| changePassword | Prompt MATLAB Job Scheduler password change |
|---|---|
| demote | Demote job in cluster queue |
| promote | Promote job in MATLAB Job Scheduler cluster queue |
MJSComputeCloud
Local
| getDebugLog | Read output messages from job run in CJS cluster |
|---|
HPC Server, PBS Pro, LSF, TORQUE, Slurm, and Generic
Properties
Common to All Cluster Types
The following properties are common to all cluster object types.
| Property | Description |
|---|---|
| ClusterMatlabRoot | Specifies path to MATLAB for workers to use |
| Host | Host name of the cluster head node |
| JobStorageLocation | Location where cluster stores job and task information |
| Jobs | List of jobs contained in this cluster |
| LicenseNumber | License number to use when running jobs with this cluster |
| Modified | True if any properties in this cluster have been modified |
| NumThreads | Number of computational threads for workers |
| NumWorkers | Number of workers available for this cluster |
| OperatingSystem | Operating system of nodes used by cluster |
| PreferredPoolNumWorkers | Preferred number of workers in a parallel pool |
| Profile | Profile used to build this cluster |
| RequiresOnlineLicensing | true if the cluster is using online licensing |
| Type | Type of this cluster |
| UserData | Information associated with cluster object within client session |
Specify the JobStorageLocation property as one of the following.
- If you use the generic scheduler interface in remote mode to interact with a third-party scheduler, and the client and workers use different operating systems, specify a structure. The structure must have the fields
'windows'and'unix'. The fields are the Windows and UNIX® path corresponding to the folder where the cluster stores job and task information. The following structure specifies the same folder using a Windows UNC path ('\\organization\some\path') and a UNIX path ('/organization/some/path'):
struct('windows', '\organization\some\path', 'unix', '/organization/some/path') - Otherwise, use a character vector or string to specify the folder where the cluster stores job and task information.
MATLAB Job Scheduler
MJS cluster objects have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| AllHostAddresses | IP addresses of the cluster host |
| BusyWorkers | Workers currently running tasks |
| IdleWorkers | Workers currently available for running tasks |
| HasSecureCommunication | True if cluster is using secure communication |
| MaxNumWorkers | Maximum number of workers this cluster can have. When you use a cluster without automatic resizing, this value is the number of workers in the cluster.When you use a cluster with automatic resizing, workers are added automatically up to this maximum value as necessary. For more information on automatic resizing, see Set up MATLAB Job Scheduler Cluster for Auto-Resizing (MATLAB Parallel Server). |
| Name | Name of this cluster |
| NumBusyWorkers | Number of workers currently running tasks |
| NumIdleWorkers | Number of workers available for running tasks |
| PromptForPassword | True if system should prompt for password when authenticating user |
| SecurityLevel | Degree of security applied to cluster and its jobs. For descriptions of security levels, see Set MATLAB Job Scheduler Cluster Security (MATLAB Parallel Server). |
| State | Current state of cluster |
| Username | User accessing cluster |
MJSComputeCloud
MJSComputeCloud cluster objects have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| BusyWorkers | Workers currently running tasks |
| Certificate | Cluster SSL certificate |
| HasSecureCommunication | true if cluster is using secure communication |
| Identifier | Unique cluster identifier |
| IdleWorkers | Workers currently available for running tasks |
| MatlabVersion | Version of MATLAB running on the workers |
| MaxNumWorkers | Maximum number of workers this cluster can have. When you use a cluster without automatic resizing, this value is the number of workers when you started the cluster.When you use a cluster with automatic resizing, workers are added automatically up to this maximum value as necessary. For more information on automatic resizing, see Resize Clusters Automatically. |
| Name | Name of this cluster |
| NumBusyWorkers | Number of workers currently running tasks |
| NumIdleWorkers | Number of workers available for running tasks |
| NumWorkersRequested | Number of workers requested for this cluster. The cluster adds or removes workers as soon as possible to reach this number. |
| SharedState | The shared state of the cluster, which can be one these options: Personal – Only you can use this cluster, so long as you created it.Shareable – Anyone can use this cluster. |
| ShutdownAt | Shutdown time or event |
| State | Current state of cluster |
| Username | User accessing cluster |
Local
Local cluster objects have no editable properties beyond the properties common to all clusters.
HPC Server
HPCServer cluster objects are supported on clients running Windows.
HPCServer cluster objects have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| ClusterVersion | Version of Microsoft Windows HPC Server running on the cluster |
| HasSharedFilesystem | Specify whether client and cluster nodes share job storage location |
| JobDescriptionFile | Name of XML job description file to use when creating jobs |
| JobTemplate | Name of job template to use for jobs submitted to HPC Server |
| Name | Name of this cluster |
| UseSOAJobSubmission | Allow service-oriented architecture (SOA) submission on HPC Server |
PBS Pro and TORQUE
PBSPro and Torque cluster objects have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| AdditionalProperties | Additional properties for plugin scripts (since R2024a) |
| HasSharedFilesystem | Specify whether client and cluster nodes share job storage location |
| PluginScriptsLocation | Folder containing scheduler plugin scripts (since R2024a) |
| ProcsPerNode | Number of processors per node, specified as a finite positive integer scalar.When you submit a job to the cluster, the number of cores per node that MATLAB requests is guaranteed to be less than or equal toProcsPerNode. Set ProcsPerNode equal to the maximum number of processors you want MATLAB to request from each cluster node.MATLAB requests the smallest number of cores per node required to run the job. If the NumThreads property of the cluster is less than or equal to ProcsPerNode, MATLAB requests NumThreads processors per worker, then maximizes the number of workers per node. For example ifProcsPerNode is 16 andNumThreads is 5, MATLAB requests 15 cores, the largest multiple of5 that is less thanProcsPerNode.If NumThreads of the cluster is greater thanProcsPerNode, MATLAB requests ProcsPerNode processors per node.When NumThreads is greater thanProcsPerNode, you might encounter performance issues. As a best practice, set NumThreads less than or equal toProcsPerNode. For more information, see Edit Number of Workers and Cluster Settings. |
| SubmitArguments | Specify additional arguments to use when submitting jobs. You can use this property to set the AdditionalSubmitArgs additional property. |
LSF and Slurm
LSF and Slurm cluster objects have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| AdditionalProperties | Additional properties for plugin scripts (since R2024a) |
| HasSharedFilesystem | Specify whether client and cluster nodes share job storage location |
| PluginScriptsLocation | Folder containing scheduler plugin scripts (since R2024a) |
| SubmitArguments | Specify additional arguments to use when submitting jobs. You can use this property to set the AdditionalSubmitArgs additional property. |
Generic
Generic cluster objects are supported on clients running Windows, macOS, or Linux®.
If you create a Generic cluster object from an R2017a or later profile, you have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| AdditionalProperties | Additional properties for plugin scripts |
| HasSharedFilesystem | Specify whether client and cluster nodes share job storage location |
| PluginScriptsLocation | Folder containing scheduler plugin scripts |
If you create a Generic cluster object from an R2016b or earlier profile, you have the following properties in addition to the common properties:
| Property | Description |
|---|---|
| CancelJobFcn | Function to run when cancelling job |
| CancelTaskFcn | Function to run when cancelling task |
| CommunicatingSubmitFcn | Function to run when submitting communicating job |
| DeleteJobFcn | Function to run when deleting job |
| DeleteTaskFcn | Function to run when deleting task |
| GetJobStateFcn | Function to run when querying job state |
| HasSharedFilesystem | Specify whether client and cluster nodes share job storage location |
| IndependentSubmitFcn | Function to run when submitting independent job |
Help
For further help on cluster objects, including links to help for specific cluster types and object properties, at the command line, type:
Version History
Introduced in R2012a
R2024a: ResourceTemplate property has been removed
The ResourceTemplate property of the LSF,Slurm, PBSPro and TORQUE cluster objects has been removed. To control resource allocation, modify theindependentSubmitFcn.m andcommunicatingSubmitFcn.m functions located in thePluginScriptsLocation instead.