GPUDevice - Query or select a GPU device - MATLAB (original) (raw)

Query or select a GPU device

Description

A GPUDevice object represents a graphic processing unit (GPU) in your computer. You can use the GPU to run MATLAB® code that supports gpuArray variables or execute CUDA® kernels using CUDAKernel objects.

You can use a GPUDevice object to inspect the properties of your GPU device, reset the GPU device, or wait for your GPU to finish executing a computation. To obtain a GPUDevice object, use the gpuDevice function. You can also select or deselect your GPU device using the gpuDevice function. If you have access to multiple GPUs, use the gpuDevice function to choose a specific GPU device on which to execute your code.

You do not need to use a GPUDevice object to run functions on a GPU. For more information on how to use GPU-enabled functions, see Run MATLAB Functions on a GPU.

Creation

Syntax

Description

gpuDevice displays the properties of the currently selected GPU device. If there is no currently selected device, gpuDevice selects the default device without clearing it. Use this syntax when you want to inspect the properties of your GPU device.

D = gpuDevice returns a GPUDevice object representing the currently selected device. If there is no currently selected device,gpuDevice selects the default device and returns aGPUDevice object representing that device without clearing it.

example

D = gpuDevice([ind](#mw%5F296d3639-89a0-4e0e-ba85-452e14bf1b1a)) selects the GPU device specified by index ind. If the specified GPU device is not supported, an error occurs. This syntax resets the specified device and clears its memory, even if the device is already currently selected (equivalent to the reset function). All workspace variables representinggpuArray or CUDAKernel variables are now invalid and must be cleared from the workspace or redefined.

example

gpuDevice([]), with an empty argument (as opposed to no argument), deselects the GPU device and clears its memory of gpuArray and CUDAKernel variables. This syntax leaves no GPU device selected as the current device.

example

Input Arguments

expand all

ind — Index of the GPU device

integer

Index of the GPU device, specified as an integer in the range 1 to gpuDeviceCount.

Example: gpuDevice(1);

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

Properties

expand all

Identity

Name — Name of the GPU device

character array

This property is read-only.

Name of the GPU device, specified as a character array. The name assigned to the device is derived from the GPU device model.

Data Types: char

Index — Index of the GPU device

integer

This property is read-only.

Index of the GPU device, specified as an integer in the range 1 to gpuDeviceCount. Use this index to select a particular GPU device.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

UUID — Universally unique identifier

character array

Since R2024a

This property is read-only.

Universally unique identifier (UUID) of the device, specified as a character array.

The UUID typically starts with 'GPU-' and includes a 36-character hexadecimal sequence. You can use the UUID to distinguish otherwise identical GPUs.

Data Types: char

State

DeviceSelected — Flag for currently selected device

0 (false) | 1 (true)

This property is read-only.

Flag for currently selected device, specified by the logical values0 (false) or 1 (true).

Data Types: logical

DeviceAvailable — Flag for available device

0 (false) | 1 (true)

This property is read-only.

Flag for available device, specified by the logical values 0 (false) or 1 (true). This property indicates whether the device is available for use in the current MATLAB session. Unsupported devices with a DeviceSupported property of 0 (false) are always unavailable. A device can also be unavailable if its ComputeMode property is set to 'Exclusive thread', 'Exclusive process', or'Prohibited'.

Data Types: logical

DeviceSupported — Flag for supported device

0 (false) | 1 (true)

This property is read-only.

Flag for supported device, specified by the logical values 0 (false) or 1 (true). Not all devices are supported; for example, devices with insufficientComputeCapability.

Data Types: logical

LastAccessed — Date and time device was last accessed

datetime array

Since R2024b

This property is read-only.

Date and time the device was last accessed by the current MATLAB session, specified as a scalar datetime array. If the device has not been accessed in this session, then LastAccessed is Not-a-Time (NaT).

Most uses of a GPU device in MATLAB update LastAccessed, including:

Querying properties of the currently selected device does not updateLastAccessed.

Data Types: datetime

Memory

TotalMemory — Total memory

scalar

This property is read-only.

Total memory (in bytes) on the device, specified as a scalar value.

Data Types: double

AvailableMemory — Total memory available for data

scalar

This property is read-only.

Total memory (in bytes) available for data, specified as a scalar value. This property is available only for the currently selected device. This value can differ from the value reported by the NVIDIA® System Management Interface due to memory caching.

Data Types: double

CachePolicy — Current caching policy of GPU device

'balanced' | 'minimum' | 'maximum'

Since R2023a

Caching policy of the GPU device, specified as 'balanced','minimum', or 'maximum'. The caching policy determines how much GPU memory can be cached to accelerate computation, specified as one of the following values.

The default value is 'balanced' for devices in'Default' or 'Prohibited' compute mode and'maximum' for devices in 'Exclusive process' compute mode. For more information on the compute mode property, see[ComputeMode](parallel.gpu.gpudevice.html#mw%5Fd0710152-2f05-4d33-b8e1-b5628e50cb67).

Note

Data Types: char | string

Driver

GraphicsDriverVersion — Graphics driver version in use

character array

Since R2023a

This property is read-only.

Graphics driver version currently in use by the GPU device, specified as a character array.

Download the latest graphics driver for your GPU at NVIDIA Driver Downloads.

Data Types: char

DriverModel — Operating model of graphics driver

'WDDM' | 'TCC' | 'N/A'

Since R2023a

This property is read-only.

Operating model of the graphics driver, specified as one of these values:

For more information about changing models and which GPU devices support'TCC', see the NVIDIA documentation.

Data Types: char

ComputeMode — Compute mode

'Default' | 'Exclusive process' | 'Prohibited'

This property is read-only.

Compute mode of the device, specified as one of the following values.

'Default' The device is not restricted, and multiple applications can use it simultaneously. MATLAB can share the device with other applications, including other MATLAB sessions or workers.
'Exclusive process' Only one application at a time can use the device. While the device is selected in MATLAB, other applications cannot use it, including other MATLAB sessions or workers.
'Prohibited' The device cannot be used.

For more information changing the compute mode of your GPU device, consult the NVIDIA documentation.

Data Types: char

KernelExecutionTimeout — Flag for timeout for long-running kernels

0 (false) | 1 (true)

This property is read-only.

Flag for timeout for long-running kernels, specified as 0 (false) or 1 (true). IfKernelExecutionTimeout is 1 (true), then the operating system places an upper bound on the time allowed for the CUDA kernel to execute. After this time, the CUDA driver times out the kernel and returns an error.

Data Types: logical

Capabilities

ComputeCapability — Computational capability of the GPU device

character array

This property is read-only.

Computational capability of the GPU device, specified as a character array. To use the selected GPU device in MATLAB, ComputeCapability must meet the required specification in GPU Computing Requirements.

Data Types: char

MultiprocessorCount — Number of streaming multiprocessors

scalar

This property is read-only.

Number of streaming multiprocessors present on the device, specified as a scalar value.

ClockRateKHz — Peak clock rate

scalar

This property is read-only.

Peak clock rate of the GPU in kHz, specified as a scalar value.

Data Types: double

SingleDoubleRatio — Ratio of single- to double-precision FPUs

scalar

Since R2024b

This property is read-only.

Ratio of single- to double-precision floating point units (FPUs) on the device, specified as a scalar value.

The ratio indicates the single-precision processing power relative to the double-precision processing power of the device. Devices that are suitable for double-precision computations, such as solving linear systems, generally have a lower ratio. Devices that are suitable for single-precision computations, such as training deep neural networks and rendering graphics, generally have a larger ratio.

Data Types: double

Kernel Programming

MaxThreadsPerBlock — Maximum supported number of threads per block

scalar

This property is read-only.

Maximum supported number of threads per block during CUDAKernel execution, specified as a scalar value.

Example: 1024

Data Types: double

MaxShmemPerBlock — Maximum supported amount of shared memory

scalar

This property is read-only.

Maximum supported amount of shared memory that a thread block can use duringCUDAKernel execution, specified as a scalar value.

Data Types: double

MaxThreadBlockSize — Maximum size in each dimension for thread block

vector

This property is read-only.

Maximum size in each dimension for thread block, specified as a vector. Each dimension of a thread block must not exceed these dimensions. Also, the product of the thread block size must not exceed MaxThreadsPerBlock.

Example: [1024 1024 64]

Data Types: double

MaxGridSize — Maximum size of grid of thread blocks

vector

This property is read-only.

Maximum size of grid of thread blocks, specified as a vector.

Example: [2.1475e+09 65535 65535]

Data Types: double

SIMDWidth — Number of simultaneously executing threads

scalar

This property is read-only.

Number of simultaneously executing threads, specified as a scalar value.

Data Types: double

ToolkitVersion — CUDA toolkit version

scalar

This property is read-only.

CUDA toolkit version used by the current release of MATLAB, specified as a scalar value.

Data Types: double

Object Functions

The following functions are also available:

parallel.gpu.GPUDevice.isAvailable(ind) Returns logical 1 or true if the GPU specified by index ind is supported and capable of being selected. ind can be an integer or a vector of integers; the default index is the current device.
parallel.gpu.GPUDevice.getDevice(ind) Returns a GPUDevice object without selecting it.

For a complete list of functions, use the methods function on theGPUDevice object:

methods('parallel.gpu.GPUDevice')

You can get help on any of the object functions with the following command:

help parallel.gpu.GPUDevice.functionname

where functionname is the name of the function. For example, to get help on isAvailable, type:

help parallel.gpu.GPUDevice.isAvailable

Examples

collapse all

Identify and Select a GPU Device

This example shows how to use gpuDevice to identify and select which device you want to use.

To determine how many GPU devices are available in your computer, use the gpuDeviceCount function.

gpuDeviceCount("available")

When there are multiple devices, the first is the default. You can examine its properties with the gpuDeviceTable function to determine if that is the one you want to use.

ans=2×5 table Index Name ComputeCapability DeviceAvailable DeviceSelected _____ __________________ _________________ _______________ ______________

  1      "NVIDIA RTX A5000"          "8.6"               true              false     
  2      "Quadro P620"               "6.1"               true              false

If the first device is the device you want to use, you can proceed. To run computations on the GPU, use gpuArray enabled functions. For more information, see Run MATLAB Functions on a GPU.

To verify that MATLAB® can use your GPU, use the canUseGPU function. The function returns 1 (true) if there is a GPU available for computation and 0 (false) otherwise.

To diagnose an issue with your GPU setup, for example if canUseGPU returns 0 (false), use the validateGPU function. Validating your GPU is optional.

Beginning GPU validation

Performing system validation

CUDA-supported platform .................................................PASSED

CUDA-enabled graphics driver exists .....................................PASSED

Version: 537.70

CUDA-enabled graphics driver load .......................................PASSED

CUDA environment variables ..............................................PASSED

CUDA device count .......................................................PASSED

Found 2 devices.

GPU libraries load ......................................................PASSED

Performing device validation for device index 1

Device exists ...........................................................PASSED

NVIDIA RTX A5000

Device supported ........................................................PASSED

Device available ........................................................PASSED

Device is in 'Default' compute mode.

Device selectable .......................................................PASSED

Device memory allocation ................................................PASSED

Device kernel launch ....................................................PASSED

Finished GPU validation with no failures.

To use another device, call gpuDevice with the index of the other device.

ans = CUDADevice with properties:

             Name: 'Quadro P620'
            Index: 2 (of 2)
ComputeCapability: '6.1'
      DriverModel: 'WDDM'
      TotalMemory: 2147352576 (2.15 GB)
  AvailableMemory: 1596066816 (1.60 GB)
  DeviceAvailable: true
   DeviceSelected: true

Show all properties.

Alternatively, you can determine how many GPU devices are available, inspect some of their properties, and select a device to use from the MATLAB® desktop. On the Home tab, in the Environment area, select Parallel > Select GPU Environment.

gpuDevice2.png

Query Compute Capabilities

Create an object representing the default GPU device and query its compute capability.

D = gpuDevice; D.ComputeCapability

Query the compute capabilities of all available GPU devices.

for idx = 1:gpuDeviceCount D = gpuDevice(idx); fprintf(1,"Device %i has ComputeCapability %s \n", ... D.Index,D.ComputeCapability) end

Device 1 has ComputeCapability 8.6 Device 2 has ComputeCapability 6.1

Compare the compute capabilities and availability of the GPU devices in your system using gpuDeviceTable.

ans=2×5 table Index Name ComputeCapability DeviceAvailable DeviceSelected _____ __________________ _________________ _______________ ______________

  1      "NVIDIA RTX A5000"          "8.6"               true              false     
  2      "Quadro P620"               "6.1"               true              true

Query and Change Caching Policy

Change the caching policy of your GPU.

Create an object representing the default GPU device.

D = CUDADevice with properties:

             Name: 'NVIDIA RTX A5000'
            Index: 1 (of 2)
ComputeCapability: '8.6'
      DriverModel: 'TCC'
      TotalMemory: 25544294400 (25.54 GB)
  AvailableMemory: 25120866304 (25.12 GB)
  DeviceAvailable: true
   DeviceSelected: true

Show all properties.

Access the CachePolicy property of the GPU device.

Change the caching policy to allow the GPU to cache the maximum amount of memory for accelerating computation.

D.CachePolicy = "maximum"; D.CachePolicy

Reset the caching policy to the default policy by setting the property to [].

Calling reset(D) or selecting another device with gpuDevice also resets the caching policy to its default value.

Use Multiple GPUs in Parallel Pool

If you have access to several GPUs, you can perform your calculations on multiple GPUs in parallel using a parallel pool.

To determine the number of GPUs that are available for use in MATLAB, use the gpuDeviceCount function.

availableGPUs = gpuDeviceCount("available")

Start a parallel pool with as many workers as available GPUs. For best performance, MATLAB assigns a different GPU to each worker by default.

parpool("Processes",availableGPUs);

Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 3).

To identify which GPU each worker is using, call gpuDevice inside an spmd block. The spmd block runs gpuDevice on every worker.

Use parallel language features, such as parfor or parfeval, to distribute your computations to workers in the parallel pool. If you use gpuArray enabled functions in your computations, these functions run on the GPU of the worker. For more information, see Run MATLAB Functions on a GPU. For an example, see Run MATLAB Functions on Multiple GPUs.

When you are done with your computations, shut down the parallel pool. You can use the gcp function to obtain the current parallel pool.

If you want to use a different choice of GPUs, then you can use gpuDevice to select a particular GPU on each worker, using the GPU device index. You can obtain the index of each GPU device in your system using the gpuDeviceCount function.

Suppose you have three GPUs available in your system, but you want to use only two for a computation. Obtain the indices of the devices.

[availableGPUs,gpuIndx] = gpuDeviceCount("available")

Define the indices of the devices you want to use.

Start your parallel pool. Use an spmd block and gpuDevice to associate each worker with one of the GPUs you want to use, using the device index. The spmdIndex function identifies the index of each worker.

parpool("Processes",numel(useGPUs));

Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 2).

spmd gpuDevice(useGPUs(spmdIndex)); end

As a best practice, and for best performance, assign a different GPU to each worker.

When you are done with your computations, shut down the parallel pool.

Extended Capabilities

Thread-Based Environment

Run code in the background using MATLAB® backgroundPool or accelerate code with Parallel Computing Toolbox™ ThreadPool.

This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.

Version History

Introduced in R2010b

expand all

R2024b: New Device Properties and Updated Display

Use the gpuDevice function to inspect these new properties of your GPU device:

Creating or querying a GPUDevice object now displays only theName, Index,ComputeCapability, DriverModel,TotalMemory, AvailableMemory,DeviceAvailable, and DeviceSelected properties. To view all of the properties of a device, create or query a GPUDevice object without suppressing output and click the Show all properties link.

D =

CUDADevice with properties:

             Name: 'NVIDIA RTX A5000'
            Index: 1 (of 2)
ComputeCapability: '8.6'
      DriverModel: 'TCC'
      TotalMemory: 25544294400 (25.54 GB)
  AvailableMemory: 25120866304 (25.12 GB)
  DeviceAvailable: true
   DeviceSelected: true

Show all properties.

The SupportsDouble, GPUOverlapsTransfers, andCanMapHostMemory properties are no longer displayed but you can still query these properties using dot notation. There are no plans to remove these properties.

R2024a: Identify GPUs using their UUIDs

Inspect the universally unique identifier (UUID) of your GPU using theUUID property of a GPUDevice object. You can use the UUID to distinguish otherwise identical GPUs.

For example, you can inspect the UUID of two GPUs using the gpuDeviceTable function.

gpuDeviceTable(["Index","Name","UUID"])

Index           Name                              UUID                   
_____    __________________    __________________________________________

  1      "NVIDIA RTX A5000"    "GPU-957b509e-322a-ae88-59c8-b7435d0f98f4"
  2      "NVIDIA RTX A5000"    "GPU-6f3ad2c0-5ea1-b1a2-1dca-cd756d10dbc0"
  3      "NVIDIA RTX A5000"    "GPU-41c24f34-c915-919b-0bb3-20d07117e0ec"
  4      "NVIDIA RTX A5000"    "GPU-23ab01ce-2f42-2d0c-0f6b-db7ac8c10867"

Alternatively, you can select a GPU using the gpuDevice function and query its UUID.

'GPU-957b509e-28ca-ae88-59c8-b7435d0f98f4'

R2023a: Changes to Device Properties