Overview of creating an instance with attached GPUs (original) (raw)

Linux Windows

Compute Engine provides graphics processing units (GPUs) that you can add to your virtual machines (VMs). You can use these GPUs to accelerate specific workloads on your VMs such as machine learning and data processing.

This document provides an overview of the steps required for creating a VM with attached GPUs.

For more information about GPUs on Compute Engine, seeAbout GPUs.

Select the GPU model

For a list of GPU models that are available, seeGPU platforms. Also make a note of the machine type that is supported for the selected GPU model.

For each model, it might also be helpful to review the following:

• Supported regions and zones.
• GPU pricing to understand the cost to use each GPU model on your VMs. For VMs that use accelerator-optimized machines, also review VM instance pricing.

Limitations

In addition to therestrictions for all VMs with GPUs, each machine series with attached GPUs has the following limitations:

A4 VMs

• You can only request capacity by using thesupported provisioning optionsfor an A4 machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A4 machine type.
• You can only use an A4 machine type in certain regions and zones.
• You can't use Persistent Disk (regional or zonal) on a VM that use A4 machine type.
• The A4 machine type is only available on the Emerald Rapids CPU platform.
• You can't change the machine type of an A4 machine type VM. If you need to change the machine type, you must create a new VM.
• You can't change the machine type of a VM to an A4 machine type. If you need a VM that uses an A4 machine type, you must create a new VM.
• A4 machine types don't supportsole-tenancy.
• You can't run Windows operating systems on an A4 machine type.

A3 Ultra VMs

• You can only request capacity by using thesupported provisioning optionsfor an A3 Ultra machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A3 Ultra machine type.
• You can only use an A3 Ultra machine type in certain regions and zones.
• You can't use Persistent Disk (regional or zonal) on a VM that use A3 Ultra machine type.
• The A3 Ultra machine type is only available on the Emerald Rapids CPU platform.
• You can't change the machine type of an A3 Ultra machine type VM. If you need to change the machine type, you must create a new VM.
• You can't change the machine type of a VM to an A3 Ultra machine type. If you need a VM that uses an A3 Ultra machine type, you must create a new VM.
• A3 Ultra machine types don't supportsole-tenancy.
• You can't run Windows operating systems on an A3 Ultra machine type.

A3 Mega VMs

• You can only request capacity by using thesupported provisioning optionsfor an A3 Mega machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A3 Mega machine type.
• You can only use an A3 Mega machine type in certain regions and zones.
• You can't use regional Persistent Disk on a VM that use A3 Mega machine type.
• The A3 Mega machine type is only available on the Sapphire Rapids CPU platform.
• You can't change the machine type of an A3 Mega machine type VM. If you need to change the machine type, you must create a new VM.
• You can't change the machine type of a VM to an A3 Mega machine type. If you need a VM that uses an A3 Mega machine type, you must create a new VM.
• A3 Mega machine types don't supportsole-tenancy.
• You can't run Windows operating systems on an A3 Mega machine type.

A3 High VMs

• You can only request capacity by using thesupported provisioning optionsfor an A3 High machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A3 High machine type.
• You can only use an A3 High machine type in certain regions and zones.
• You can't use regional Persistent Disk on a VM that uses A3 High machine type.
• The A3 High machine type is only available on the Sapphire Rapids CPU platform.
• You can't change the machine type of an A3 High machine type VM. If you need to change the machine type, you must create a new VM.
• You can't change the machine type of a VM to an A3 High machine type. If you need a VM that uses an A3 High machine type, you must create a new VM.
• A3 High machine types don't supportsole-tenancy.
• You can't run Windows operating systems on an A3 High machine type.
• For a3-highgpu-1g, a3-highgpu-2g, anda3-highgpu-4gmachine types, the following limitations apply:
  • For these machine types, you must create VMs using Spot VMs or a feature that uses theDynamic Workload Scheduler (DWS), such as resize requests in a MIG. For detailed instructions on either of these options, review the following:
    * To create Spot VMs, set the provisioning model to SPOT when youCreate an accelerator-optimized VM.
    * To create a resize request in a MIG, which uses DWS, seeCreate a MIG with GPU VMs.
  • You can't use Hyperdisk Balanced with these machine types.
  • You can't create reservations.
    If you try to create a VM by using standard provisioning or try to create a reservation for these machine types, you'll receive anerror message.

A3 Edge VMs

• You can only request capacity by using thesupported provisioning optionsfor an A3 Edge machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A3 Edge machine type.
• You can only use an A3 Edge machine type in certain regions and zones.
• You can't use regional Persistent Disk on a VM that use A3 Edge machine type.
• The A3 Edge machine type is only available on the Sapphire Rapids CPU platform.
• You can't change the machine type of an A3 Edge machine type VM. If you need to change the machine type, you must create a new VM.
• You can't change the machine type of a VM to an A3 Edge machine type. If you need a VM that uses an A3 Edge machine type, you must create a new VM.
• A3 Edge machine types don't supportsole-tenancy.
• You can't run Windows operating systems on an A3 Edge machine type.

A2 Standard VMs

• You can only request capacity by using thesupported provisioning optionsfor an A2 Standard machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A2 Standard machine type.
• You can only use an A2 Standard machine type in certain regions and zones.
• The A2 Standard machine type is only available on the Cascade Lake platform.
• If your VM uses an A2 Standard machine type, you can only switch from one A2 Standard machine type to another A2 Standard machine type. You can't change to any other machine type. For more information, seeModify accelerator-optmized VMs.
• You can't use the Windows operating system with <codea2-megagpu-16g< code="" translate="no" dir="ltr"> A2 Standard machine types. When using Windows operating systems, choose a different A2 Standard machine type.</codea2-megagpu-16g<>
• You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Standard machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.
• A2 Standard machine types don't supportsole-tenancy.

A2 Ultra VMs

• You can only request capacity by using thesupported provisioning optionsfor an A2 Ultra machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an A2 Ultra machine type.
• You can only use an A2 Ultra machine type in certain regions and zones.
• The A2 Ultra machine type is only available on the Cascade Lake platform.
• If your VM uses an A2 Ultra machine type, you can't change the machine type. If you need to use a different A2 Ultra machine type, or any other machine type, you must create a new VM.
• You can't change any other machine type to an A2 Ultra machine type. If you need a VM that uses an A2 Ultra machine type, you must create a new VM.
• You can't do a quick format of the attached Local SSDs on Windows VMs that use A2 Ultra machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.

G2 VMs

• You can only request capacity by using thesupported provisioning optionsfor an G2 machine type.
• You don't receive sustained use discounts andflexible committed usediscounts for VMs that use an G2 machine type.
• You can only use an G2 machine type in certain regions and zones.
• The G2 machine type is only available on the Cascade Lake platform.
• Standard Persistent Disk (pd-standard) is not supported on VMs that use G2 standard machine types. For supported disk types, seeSupported disk types for G2.
• You can't create Multi-Instance GPUs on G2 machine types.
• If you need to change the machine type of a G2 VM, reviewModify accelerator-optmized VMs.
• You can't use Deep Learning VM Images as boot disks for your VMs that use G2 machine types.
• The current default driver for Container-Optimized OS doesn't support L4 GPUs running on G2 machine types. Container-Optimized OS also only support a select set of drivers. If you want to use Container-Optimized OS on G2 machine types, review the following notes:
  • Use a Container-Optimized OS version that supports the minimum recommended NVIDIA driver version 525.60.13 or later. For more information, review theContainer-Optimized OS release notes.
  • When you install the driver, specify the latest available version that works for the L4 GPUs. For example, sudo cos-extensions install gpu -- -version=525.60.13.
• You must use the Google Cloud CLI or REST tocreate G2 VMsfor the following scenarios:
  • You want to specify custom memory values.
  • You want to customize the number of visible CPU cores.

N1+GPU VMs

To learn about the limitations for N1 instances with GPUs, seefeatures for the N1 machine seriesand GPUs for the N1 machine series.

Choose an operating system

If you are using GPUs for machine learning, use one of the following operating systems:

• Images optimized for AI workloads. You can use Ubuntu and Rocky images, which are available in accelerator-optimized versions with drivers and CUDA toolkit pre-installed. See OS imagesin the AI Hypercomputer documentation.
• Deep Learning VM Images. Each Deep Learning VM has a GPU driver installation tool and includes packages such as TensorFlow and PyTorch. You can also use a Deep Learning VM for general GPU workloads. To learn more about available images and packages installed on these images, see Choosing an image.

Alternatively, you can use any public imageor custom image. Note that some images might require a unique driver or have an install process that is out of the scope of the Compute Engine documentation. To help identify which drivers are appropriate for your OS image, see installing GPU drivers.

Check GPU quota

To protect Compute Engine systems and users, new projects have a global GPU quota, which limits the total number of GPUs you can create in any supported zone. To review GPU quota, seeGPU quota.

If you need additional GPU quota,request a quota increase. When you request GPU quota, you must request quota for the GPU types that you want to create in each region and an additional global quota for the total number of GPUs of all types in all zones.

If your project has an established billing history, it will receive quota automatically after you submit the request.

GPU VMs and preemptible allocation quotas

VMs that use thestandard provisioning modeltypically can't use preemptible allocation quotas. Preemptible quotas are for temporary workloads and are usually more available. If your project doesn't have preemptible quota, and you have never requested it, then all VMs in your project consume standard allocation quotas.

If you request preemptible allocation quota, then VMs that use the standard provisioning model must meet all of the following criteria to consume preemptible allocation quota:

• The VMs have GPUs attached.
• The VMs are configured to be automatically deleted after a predefined run time through one of the following methods:
  • The maxRunDuration or terminationTime field. For more information, seeLimit the run time of a VMor Limit the run time of VMs in a MIG.
  • The requestedRunDuration field, which is only available for managed instance groups (MIGs). For more information, seeAbout resize requests in a MIG.
• The VM is not allowed to consume reservations. For more information, seePrevent compute instances from consuming reservations.

When you consume preemptible allocation for time-bound GPU workloads, you can benefit from both uninterrupted run time and the high obtainability of preemptible allocation quota. For more information, seePreemptible quotas.

Create a VM that has attached GPUs

To create a VM that has attached GPUs, complete the following steps:

1. Create the VM. The method used to create a VM depends on the GPU model selected.
   • To create a VM that has attached NVIDIA B200 or H200 GPUs, seeCreate an A3 Ultra or A4 VM.
   • To create a VM that has attached NVIDIA H100, A100, or L4 GPUs, seeCreate an A3, A2, or G2 VM.
   • To create a VM that has attached NVIDIA T4, P4, P100, or V100 GPUs, seeCreate an N1 VM that has attached GPUs.
2. For the VM to use the GPU, you need toinstall the GPU driver on your VM. If you enabled an NVIDIA RTX virtual workstation (formerly known as NVIDIA GRID),install a driver for virtual workstation.

What's next?

• Learn more aboutGPU platforms.
• Learn more about thefeatures and limitations of using GPUs.
• Learn how toview the actual and forecasted usage of your GPUs.