Google Cloud Hyperdisk overview (original) (raw)

This document describes the features of Google Cloud Hyperdisk. Hyperdisk is the fastest and most efficient durable disk for Compute Engine. If you need boot or data disks for your compute instances—virtual machine (VM) instances, containers, and bare metal instances—Google recommends using Hyperdisk.

For information about the other block storage options in Compute Engine, see Choose a disk type.

To create a new Hyperdisk volume, seeCreate a Hyperdisk volume.

Hyperdisk features

With Hyperdisk you can provision, manage, and scale your Compute Engine workloads without the cost and complexity of a typical on-premises storage area network (SAN).

Hyperdisk volumes have the following features:

• Function as physical disks: you can use a Hyperdisk volume with a compute instance as if it were a physical disk attached to the instance. When you read to or write from a Hyperdisk volume, data is transmitted over the network.
• Higher performance: Hyperdisk offers higher IOPS and throughput than Persistent Disk by leveraging Google's Titanium storage offload technology.
• Customizable performance: you can choose the performance—IOPS and/or throughput—of each Hyperdisk volume. You can also increase or decrease a Hyperdisk volume's performance while it's in use.
• Support for high availability: in the unlikely event of a zonal or regional outage, you can ensure high availability for your data by enabling one or both of the following features:
  • To protect data your data in case of a zonal outage, useHyperdisk Balanced High Availability. Data on Hyperdisk Balanced High Availability volumes is synchronously replicated across two zones within the same region to protect against up to one zonal outage.
  • To protect your data from a regional outage, maintain a replica of your data in another region by using Asynchronous Replication. When you enable Asynchronous Replication for a disk, data in one region is continuously copied to a replica in a secondary region. If a regional outage occurs, you canfailoveryour data to a secondary region. Asynchronous Replication is available for Hyperdisk Balanced, Hyperdisk Balanced High Availability, and Hyperdisk Extreme volumes.
• Portability: you can change the compute instance that a Hyperdisk volume is attached to.
• Shareable between VMs: for high availability workloads, certain Hyperdisk types can be shared by multiple VMs. Each VM has simultaneous read-write or read-only access to the volume.
• Support for pooled capacity and performance: to simplify planning, avoid overprovisioning storage, and reduce costs, you can purchase Hyperdisk storage and performance in bulk by using Hyperdisk Storage Pools.

Choose a Hyperdisk type for your workload

To add Hyperdisk volumes to your workloads, you must choose a Hyperdisk type. Each Hyperdisk type is designed and optimized for a specific type of workload. The following is a list of the available Hyperdisk types.

• Hyperdisk Balanced
• Hyperdisk Balanced High Availability
• Hyperdisk Extreme
• Hyperdisk Throughput
• Hyperdisk ML

For most workloads, we recommend Hyperdisk Balanced.

To select a Hyperdisk type, compare your workload's type and its performance requirements with the information in the following table. For detailed information about a specific Hyperdisk type, see the linked page in the Recommended Hyperdisk type column.

1 You can't specify a throughput level for Hyperdisk Extreme volumes. The provisioned throughput is based on the IOPS level you specify.
2 You can't specify an IOPS level for Hyperdisk Throughput and Hyperdisk ML volumes. The provisioned IOPS is based on the throughput level you specify.

Hyperdisk performance

The following is a summary of key Hyperdisk performance concepts:

• You can configure the performance (IOPS and/or throughput) limit and size of each Hyperdisk volume. You can also increase or decrease a Hyperdisk volume's performance without changing its size.
• The performance limit you specify is referred to as the provisioned performance. The provisioned performance isn't the expected performance, rather, it's the maximum performance the disk can achieve.
• The actual performance for a Hyperdisk volume is the observed performance while the volume is in use.
• For a Hyperdisk volume to reach its provisioned performance, you must attach it to a compute instance that supports the same level of performance or higher.

For a discussion of how Hyperdisk performance works, seeAbout Hyperdisk performance. For performance limits for each Hyperdisk type, seeHyperdisk performance limits.

Latency

Each Hyperdisk type has different latency profiles. Google recommends comparing Hyperdisk Throughput to the latency of a hard disk drive. You can compare the latency for Hyperdisk Balanced, Hyperdisk Balanced High Availability, Hyperdisk Extreme, and Hyperdisk ML to the latency of enterprise SSDs.

Hyperdisk Balanced and Hyperdisk Extreme offer sub-millisecond latency.

Machine series support for Hyperdisk

This section lists the machine seriesthat each Hyperdisk type supports. If a machine series doesn't support Hyperdisk, use Persistent Disk.

Select one or more machine series to see the supported Hyperdisk types.

Machine type restrictions

This section lists the restrictions that apply to the machine series that each Hyperdisk type supports.

• To use Hyperdisk Balanced with A3 VMs, the VM must have at least 8 GPUs.
• For Hyperdisk Extreme, the following restrictions apply:
  • A3 machine types require at least 4 GPUs.
  • C3 machine type require at least 88 vCPUs.
  • C3D machine types require at least 60 vCPUs.
  • C4 machine types require at least 96 vCPUs.
  • M1 machine types require at least 80 vCPUs.
  • C4A, C4D (Preview), and M3 machine types require at least 64 vCPUs.
  • M4 machine types require at least 112 vCPUs.
  • N2 requires 80 or more vCPUs; Custom N2 machine types aren't supported.
• You can't use Hyperdisk Throughput with c3-*-metal machine types.

You can share a Hyperdisk volume between multiple VMs by simultaneously attaching the same volume to multiple VMs.

The following scenarios are supported:

• Concurrent read-write access to a single volume from multiple VMs. Recommended for clustered file systems and highly available workloads like SQL Server Failover Cluster Infrastructure. Supported for Hyperdisk Balanced and Hyperdisk Balanced High Availability volumes.
• Concurrent read-only access to a single volume from multiple VMs. This is more cost effective than having multiple disks with the same data. Recommended for accelerator-optimized machine learning workloads. Supported for Hyperdisk ML volumes.

You can't attach a Hyperdisk Throughput or Hyperdisk Extreme volume to more than one VM.

To learn about disk sharing, seeShare a disk between VMs.

High availability and disaster recovery protection for Hyperdisk volumes

You can protect your data in the rare event of a zonal or regional outage by enabling replication, that is, maintaining a copy of the data in another zone or region.

Cross-zonal synchronous replication

To replicate data to another zone within the same region, you must use Hyperdisk Balanced High Availability volumes. Hyperdisk Balanced High Availability is the only supported Hyperdisk type for zonal replication.

For more information, seeAbout synchronous disk replication.

Cross-regional asynchronous replication

You can protect your data in the unlikely event of a regional outage by enabling Asynchronous Replication. Asynchronous Replication maintains a copy of the data on your volume in another region. For example, to protect a Hyperdisk Balanced volume in us-west1, you can use Asynchronous Replication to replicate the volume to a secondary volume in the us-east4 region. If the volume in us-west1became unavailable, then you could use the secondary volume in us-east4.

You can use Asynchronous Replication with the following Hyperdisk types:

• Hyperdisk Balanced
• Hyperdisk Extreme
• Hyperdisk Balanced High Availability

To learn more about cross-regional replication, see Asynchronous Replication.

Encryption for Hyperdisk volumes

By default, Compute Engine protects your Hyperdisk volumes with Google-owned and Google-managed encryption keys. You can also encrypt your Hyperdisk volumes with customer-managed encryption keys (CMEK).

For more information, see About disk encryption.

Confidential Computing with Hyperdisk volumes

You can add hardware-based encryption to a Hyperdisk Balanced disk by enabling Confidential mode for the disk when you create it. You can use Confidential mode only with Hyperdisk Balanced disks that are attached to Confidential VMs.

For more information, see Confidential mode for Hyperdisk Balanced volumes.

Durability of Hyperdisk

Compute Engine distributes the data on Hyperdisk volumes across several physical disks to ensure durability and optimize performance.

Disk durability represents the probability of data loss, by design, for a typical disk in a typical year. Hyperdisk data loss events are extremely rare and have historically been the result of coordinated hardware failures, software bugs, or a combination of the two. Google takes many steps to mitigate the industry-wide risk of silent data corruption.

Durability is calculated with a set of assumptions about hardware failures, the likelihood of catastrophic events, isolation practices and engineering processes in Google data centers, and the internal encodings used by each disk type.

Human error by a Google Cloud customer, such as when a customer accidentally deletes a disk, is outside the scope of Hyperdisk durability.

The table below shows durability for each disk type's design. 99.999% durability means that with 1,000 Hyperdisk volumes, you would likely go a hundred years without losing a single one.

Supported disk interfaces

Hyperdisk volumes are mounted as a disk on a VM using the NVMe or SCSI interface, depending on the machine type of the instance.

Hyperdisk Storage Pools

Hyperdisk Storage Pools make it easier to lower your block storage total cost of ownership and simplify block storage management. With Hyperdisk Storage Pools, you can share a pool of capacity and performance across a maximum of 1,000 disks in a single project. Because storage pools offer thin-provisioning and data reduction, you can achieve higher efficiency.

Storage pools simplify migrating your on-premises SAN to the cloud, and also make it easier to provide your workloads with the capacity and performance that they need.

You create a storage pool with the estimated capacity and performance for all workloads in a project in a specific zone. You then create disks in this storage pool and attach the disks to existing VMs. You can also create a disk in the storage pool as part of creating a new VM. Each storage pool contains one type of disk, such as Hyperdisk Throughput. There are two types of Hyperdisk Storage Pools:

• Hyperdisk Balanced Storage Pool: for general purpose workloads that are best served by Hyperdisk Balanced disks
• Hyperdisk Throughput Storage Pool: for streaming, cold data, and analytics workloads that are best served by Hyperdisk Throughput disks

For information about using Hyperdisk Storage Pools, seeAbout storage pools.

Pricing

You are billed for the total provisioned capacity of your Hyperdisk volumes until you delete them. You are charged per GiB per month. Additionally, you are billed for the following:

• Hyperdisk Balanced charges a monthly rate for the provisioned IOPS and provisioned throughput (in MiB/s) in excess of the baseline values of 3,000 IOPS and 140 MiB/s throughput.
• Hyperdisk Extreme charges a monthly rate based on the provisioned IOPS.
• Hyperdisk ML charges a monthly rate based on the provisioned throughput (in MiB/s). There is no additional charge for attaching multiple VMs to a single Hyperdisk ML volume.
• Hyperdisk Throughput charges a monthly rate based on the provisioned throughput (in MiB/s).

Because the data for regional disks is written to two locations, the cost of Hyperdisk Balanced High Availability storage is twice the cost of Hyperdisk Balanced storage.

For more pricing information, seeDisk pricing.

Hyperdisk and committed use discounts

Hyperdisk volumes are not eligible for:

• Resource-based committed use discounts (CUDs)
• Sustained use discounts (SUDs)

Hyperdisk and preemptible VM instances

Hyperdisk can be used with Spot VMs (or preemptible VMs). However, there are no discounted spot prices for Hyperdisk.

Limitations for Hyperdisk

• You can't create a machine image from a Hyperdisk volume.
• You can't back up a disk in multi-writer mode with snapshots or images. You must disable multi-writer modefirst.
• You can't create an imagefrom a Hyperdisk Extreme, Hyperdisk Throughput, or Hyperdisk Balanced High Availability volume.
• You can't create an instant snapshot from a Hyperdisk ML or Hyperdisk Throughput volume.
• You can't attach Hyperdisk Throughput or Hyperdisk Extreme volumes to more than one VM.
• Hyperdisk Extreme, Hyperdisk ML and Hyperdisk Throughput volumes can't be used as boot disks.
• You can attach a Hyperdisk ML volume to up to 100 VMs at most once every 30 seconds.
• You can't create a Hyperdisk ML disk in read-write mode from a snapshot or a disk image. You must create the disk in read-only mode.
• If you enable read-only mode for a Hyperdisk ML volume, you can't re-enable read-write mode.
• If you create a Hyperdisk Balanced volume in Confidential mode, seeadditional limitations.
• Confidential VMs with AMD SEV on C3D machine types don't support Hyperdisk Balanced and Hyperdisk Throughput.

What's next?

• Learn how to create a Hyperdisk volume.
• Learn how to clone a Hyperdisk volume.
• Learn about synchronous disk replicationwith Hyperdisk Balanced High Availability.
• Learn about Hyperdisk Storage Pools.
• Review Disk pricing information.
• Learn how to optimize performance of Hyperdisk.