Configure per VM Tier_1 networking performance (original) (raw)


Compute Engine lets you select a high-bandwidth per VM Tier_1 networking performance configuration for certain general-purpose and compute-optimized virtual machine (VM) instances. VMs with Tier_1 networking configurations are especially useful for large, distributed compute workloads with lots of heavy internode communications, such as high performance computing (HPC), machine learning (ML), and deep learning (DL).

Combining these high throughput VMs with high-performance local SSD storage is beneficial for I/O-intensive, flash-optimized databases.

Before you begin

Required roles

To get the permissions that you need to configure a VM to use per VM Tier_1 networking performance, ask your administrator to grant you the following IAM roles on your project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to configure a VM to use per VM Tier_1 networking performance. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to configure a VM to use per VM Tier_1 networking performance:

You might also be able to get these permissions with custom roles or other predefined roles.

Limitations

Bandwidth tiers

The egress bandwidth limit represents the maximum possible amount of data per unit of time (for example, gigabits per second, or Gbps) that Google Cloud allows a VM to emit from its network interfaces (NICs). The egress bandwidth includes data transferred to all Persistent Disk and Google Cloud Hyperdisk volumes attached to the VM.

Note the following about bandwidth limits:

To achieve the highest possible egress bandwidth, all of the following must be true:

For a complete discussion about egress and ingress bandwidth limits, seeNetwork bandwidth.

General-purpose C4 VMs

The following table describes the egress bandwidth limits for C4 VMs.

vCPUs Internal IP Tier_1Internal IP External IP Tier_1External IP
2 10 Gbps N/A 7 Gbps N/A
4 23 Gbps N/A 7 Gbps N/A
8 23 Gbps N/A 7 Gbps N/A
16 23 Gbps N/A 7 Gbps N/A
32 23 Gbps N/A 7 Gbps N/A
48 34 Gbps 50 Gbps 7 Gbps 25 Gbps
96 67 Gbps 100 Gbps 7 Gbps 25 Gbps
192 100 Gbps 200 Gbps 7 Gbps 25 Gbps

General-purpose C4A VMs

The following table describes the egress bandwidth limits for C4A VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
1 10 Gbps N/A 7 Gbps N/A
2 10 Gbps N/A 7 Gbps N/A
4 23 Gbps N/A 7 Gbps N/A
8 23 Gbps N/A 7 Gbps N/A
16 23 Gbps N/A 7 Gbps N/A
32 23 Gbps 50 Gbps 7 Gbps 25 Gbps
48 34 Gbps 50 Gbps 7 Gbps 25 Gbps
64 45 Gbps 75 Gbps 7 Gbps 25 Gbps
72 50 Gbps 100 Gbps 7 Gbps 25 Gbps

General-purpose C4D instances

The following table describes the egress bandwidth limits for C4D instances.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
2 10 Gbps N/A 7 Gbps N/A
4 20 Gbps N/A 7 Gbps N/A
8 20 Gbps N/A 7 Gbps N/A
16 20 Gbps N/A 7 Gbps N/A
32 23 Gbps N/A 7 Gbps N/A
48 34 Gbps 50 Gbps 7 Gbps 25 Gbps
64 45 Gbps 75 Gbps 7 Gbps 25 Gbps
96 67 Gbps 100 Gbps 7 Gbps 25 Gbps
192 100 Gbps 150 Gbps 7 Gbps 25 Gbps
384 100 Gbps 200 Gbps 7 Gbps 25 Gbps

General-purpose C3 VMs and bare metal instances

The following table describes the egress bandwidth limits for C3 VMs and bare metal instances.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
4 23 Gbps N/A 7 Gbps N/A
8 23 Gbps N/A 7 Gbps N/A
22 23 Gbps N/A 7 Gbps N/A
44 32 Gbps 50 Gbps 7 Gbps 25 Gbps
88 62 Gbps 100 Gbps 7 Gbps 25 Gbps
176 100 Gbps 200 Gbps 7 Gbps 25 Gbps
192 100 Gbps 200 Gbps 7 Gbps 25 Gbps

General-purpose C3D VMs

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
4 20 Gbps N/A 7 Gbps N/A
8 20 Gbps N/A 7 Gbps N/A
16 20 Gbps N/A 7 Gbps N/A
30 20 Gbps 50 Gbps 7 Gbps 25 Gbps
60 40 Gbps 75 Gbps 7 Gbps 25 Gbps
90 60 Gbps 100 Gbps 7 Gbps 25 Gbps
180 100 Gbps 150 Gbps 7 Gbps 25 Gbps
360 100 Gbps 200 Gbps 7 Gbps 25 Gbps

Compute-optimized C2 VMs

The following table describes the egress bandwidth limits for C2 VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
4 10 Gbps Not applicable (N/A) 7 Gbps N/A
8 16 Gbps N/A 7 Gbps N/A
16 32 Gbps N/A 7 Gbps N/A
30 32 Gbps 50 Gbps 7 Gbps 7 Gbps
60 32 Gbps 100 Gbps 7 Gbps 7 Gbps

Compute-optimized C2D VMs

The following table describes the egress bandwidth limits for C2D VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
2 10 Gbps Not applicable (N/A) 7 Gbps N/A
4 10 Gbps N/A 7 Gbps N/A
8 16 Gbps N/A 7 Gbps N/A
16 32 Gbps N/A 7 Gbps N/A
32 32 Gbps 50 Gbps 7 Gbps 25 Gbps
56 32 Gbps 50 Gbps 7 Gbps 25 Gbps
112 32 Gbps 100 Gbps 7 Gbps 25 Gbps

Compute-optimized H3 VMs

The following table describes the egress bandwidth limits for H3 VMs.

vCPUs Internal IP External IP
88 200 Gbps 1 Gbps

Memory-optimized M4 VMs

The following table describes the egress bandwidth limits for M4 VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
56 32 Gbps 50 Gbps 7 Gbps 25 Gbps
112 50 Gbps 100 Gbps 7 Gbps 25 Gbps
224 100 Gbps 200 Gbps 7 Gbps 25 Gbps

Memory-optimized M3 VMs

The following table describes the egress bandwidth limits for M3 VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
32 32 Gbps N/A 7 Gbps N/A
64 32 Gbps 50 Gbps 7 Gbps 25 Gbps
128 32 Gbps 100 Gbps 7 Gbps 25 Gbps

General-purpose N2 VMs

The following table describes the egress bandwidth limits for N2 VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
2 10 Gbps Not applicable (N/A) 7 Gbps N/A
4 10 Gbps N/A 7 Gbps N/A
8 16 Gbps N/A 7 Gbps N/A
16 32 Gbps N/A 7 Gbps N/A
32 32 Gbps 50 Gbps 7 Gbps 25 Gbps
48 32 Gbps 50 Gbps 7 Gbps 25 Gbps
64 32 Gbps 75 Gbps 7 Gbps 25 Gbps
80 32 Gbps 100 Gbps 7 Gbps 25 Gbps
96 32 Gbps 100 Gbps 7 Gbps 25 Gbps
128 32 Gbps 100 Gbps 7 Gbps 25 Gbps

General-purpose N2 (custom size shapes) VMs

The following table describes the egress bandwidth limits for custom-sized N2 VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
2, 4, or 6 10 Gbps Not applicable (N/A) 7 Gbps N/A
8, 10, 12, or 14 16 Gbps N/A 7 Gbps N/A
16, 18, 20, 22, 24, 26, 28, or 30 32 Gbps N/A 7 Gbps N/A
32, 36, 40, 44, 48, 52, 56, or 60 32 Gbps 50 Gbps 7 Gbps 25 Gbps
64, 68, 72, or 76 32 Gbps 75 Gbps 7 Gbps 25 Gbps
80 or more 32 Gbps 100 Gbps 7 Gbps 25 Gbps

General-purpose N2D VMs

The following table describes the egress bandwidth limits for N2D VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
2 10 Gbps Not applicable (N/A) 7 Gbps N/A
4 10 Gbps N/A 7 Gbps N/A
8 16 Gbps N/A 7 Gbps N/A
16 32 Gbps N/A 7 Gbps N/A
32 32 Gbps N/A 7 Gbps N/A
48 32 Gbps 50 Gbps 7 Gbps 25 Gbps
64 32 Gbps 50 Gbps 7 Gbps 25 Gbps
80 32 Gbps 50 Gbps 7 Gbps 25 Gbps
96 32 Gbps 100 Gbps 7 Gbps 25 Gbps
128 32 Gbps 100 Gbps 7 Gbps 25 Gbps
224 32 Gbps 100 Gbps 7 Gbps 25 Gbps

General-purpose N2D (custom size shapes) VMs

The following table describes the egress bandwidth limits for custom-sized N2D VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
2 or 4 10 Gbps Not applicable (N/A) 7 Gbps N/A
8 16 Gbps N/A 7 Gbps N/A
16 or 32 32 Gbps N/A 7 Gbps N/A
48, 64, or 80 32 Gbps 50 Gbps 7 Gbps 25 Gbps
96 32 Gbps 100 Gbps 7 Gbps 25 Gbps

Memory-optimized X4 instance

The following table describes the egress bandwidth limits for X4 bare metal instances.

vCPUs Internal IP External IP
960 100 Gbps 7 Gbps
1440 100 Gbps 7 Gbps
1920 100 Gbps 7 Gbps

Storage-optimized Z3 VMs

The following table describes the egress bandwidth limits for Z3 VMs.

vCPUs Internal IP Tier_1 Internal IP External IP Tier_1 External IP
88 62 Gbps 100 Gbps 7 Gbps 25 Gbps
176 100 Gbps 200 Gbps 7 Gbps 25 Gbps
192 (Preview) 100 Gbps 200 Gbps 7 Gbps 25 Gbps

Configure an instance with Tier_1 networking

You can enable Tier_1 networking when creating a compute instance if the instance doesn't use the VirtioNet interface. You can also edit an instance to add or remove Tier_1 networking, provided the instance was created with the gVNIC or IDPF network interface .

Optionally, you can alsoenable faster network packet processing with DPDKto run performance-intensive applications on a VM that uses Tier_1 networking.

Create instances and containers that use Tier_1 networking

Use the Google Cloud console, the Google Cloud CLI or REST to add Tier_1 networking to a new compute instance orcontainer.

Console

  1. In the Google Cloud console, go to the VM instances page.
    Go to VM instances
  2. Select your project.
  3. Click Create instance.
  4. Specify a Name for your compute instance. For more information, seeResource naming convention.
  5. Select a region and zone that supports the machine type you plan to use.
  6. Select a Machine configuration for your instance. To create an instance with Tier_1 networking, you must select a supported machine series and type.
    • Click the General purpose tab before selecting N2, N2D, C4, C4A, C4D, C3, or C3D from the Series drop-down menu.
    • Click the Compute optimized tab before selecting C2 or C2D from the Series menu.
    • Click the Memory optimized tab before selecting M3 or M4 from the Series menu.
    • Click the Storage optimized tab before selecting Z3 from the Series menu.
      If you get an error that the machine type is not available in the selected region, change the region to one that supports your chosen machine type.
  7. In the Machine type menu, choose a machine type that aligns with the bandwidth tier size requirements.
  8. To select a compatible operating system, in the Boot disksection, click Change, and then select a supported operating system or use the Custom Images tab to select acustom image.
  9. Optional. In the Firewall section, choose your firewall rules.
  10. Expand the Advanced options section.
  11. Expand the Networking section, and then do the following:
  12. In the Network interface card menu, select gVNIC.
  13. In the Network bandwidth section, select theEnable per VM Tier_1 networking performance checkbox.
  14. If your VM has multiple NICs or your instance uses IPv6 addresses, configure your Network interfaces.
  15. Click Create.

gcloud

Use the gcloud compute instances create commandto create a VM with a gVNIC virtual network driver. Use the --network-performance-configs flag and the--network-interface flag to configure a network performance setting for a VM. If you don't specify these flags the VM is created with the default network performance configuration.

To create a VM running container images, use thegcloud compute instances create-with-container command.

gcloud compute instances create VM_NAME
--image=OS_IMAGE
--machine-type=MACHINE_TYPE
--network-performance-configs=total-egress-bandwidth-tier=TIER_1
--network-interface=nic-type=GVNIC

Replace the following:

For example:

gcloud compute instances create instance-1
--network-performance-configs=total-egress-bandwidth-tier=TIER_1
--network-interface=nic-type=GVNIC
--image-family=rocky-linux-8-optimized-gcp
--image-project=rocky-linux-cloud
--machine-type=n2-standard-32

REST

Call the Compute Engine APIinstances.insert methodto create a VM with a high-bandwidth network configuration. Within the request body:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances

{ "name": VM_NAME, "description": string, ... "networkPerformanceConfig": { "totalEgressBandwidthTier": TIER_1 }, "networkInterfaces": [ { "nicType": "GVNIC" }, ... ] }

Replace the following:

Update a VM to include Tier_1 networking

Refer to the Updating instance propertiesdocumentation to verify that you are meeting all the requirements to successfully update your VM. Use the Google Cloud console, the Google Cloud CLI or REST to update a VM.

You can modify an existing VM to change the network configuration to include or exclude per VM Tier_1 networking performance. Your VM must already have a gVNIC interface associated with it; you can't edit your VM to add a network interface. To update the network configuration, you must stop and restart the VM.

Console

  1. In the Google Cloud console, go to the VM instances page.
    Go to VM instances
  2. Select your project.
  3. Click the name of the VM that you want to modify.
  4. Click Stopto stop the VM. If there is no Stop option, clickMore actions > Stop.
  5. Click Edit.
  6. If your VM was originally configured with a gVNIC card, select theEnable per VM Tier_1 networking performance checkbox to add per VM Tier_1 networking performance, or deselect the checkbox to remove this feature from your VM.
  7. Save your changes.
  8. Restart your VM.

gcloud

  1. Export your VM's information to a YAML file using thegcloud compute instances export command.
    gcloud compute instances export VM_NAME \
    --zone=ZONE --destination=PATH_TO_FILE
    Replace the following:
    • VM_NAME: the name of the VM
    • ZONE: the name of the zone where the VM is located
    • PATH_TO_FILE: the relative path to the YAML file
      For example:
      gcloud compute instances export instance-1 \
      --zone=europe-west1-c --destination=test-file.yaml
  2. Use the Cloud Shell Editor, or the editor of your choice to open the YAML file you created.
  3. In the file, locate the configuration section fornetworkPerformanceConfig. Change the setting fortotalEgressBandwidthTier as shown in the following example:
    networkPerformanceConfig:
    totalEgressBandwidthTier: TIER_1
    Setting totalEgressBandwidthTier to TIER_1 adds Tier_1 networking. Setting it to DEFAULT removes the configuration.
  4. Use the gcloud compute instance update-from-file commandto update the VM with the changes in the file.
    gcloud compute instances update-from-file VM_NAME \
    --zone=ZONE \
    --source=PATH_TO_FILE \
    --most-disruptive-allowed-action=RESTART
    Replace the following:
    • VM_NAME: the name of the VM
    • ZONE: the name of the zone where the VM is located
    • PATH_TO_FILE: your YAML filename
      The --most-disruptive-allowed-action=RESTART flag setting automatically restarts your VM with the updated configuration.

REST

Call theinstances.update methodto modify the network configuration.

PUT https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID?most_disruptive_allowed_action=RESTART

{ "networkPerformanceConfig":{ "totalEgressBandwidthTier": "TIER_1" }, ... }

Setting totalEgressBandwidthTier to TIER_1 adds Tier_1 networking. Setting it to DEFAULT removes the configuration.

Replace the following:

The most_disruptive_allowed_action=RESTART query parameter automatically restarts your VM with the updated configuration.

Verify high-bandwidth configuration in a VM

Use the Google Cloud console, the Google Cloud CLI or REST to generate a description of an existing VM or an existing VM running container images to verify the VM's bandwidth tier.

Console

  1. In the Google Cloud console, go to the VM instances page.
    Go to VM instances
  2. Select your project and click Continue.
  3. Click the VM name to see its configuration details and see if the VM uses per VM Tier_1 networking performance.

gcloud

Use thegcloud compute instances describe commandto check if your VM uses per VM Tier_1 networking performance.

For example:

gcloud compute instances describe VM_NAME
--format="text(name, networkPerformanceConfig)"

The output is similar to the following:

name: instance-1 networkPerformanceConfig.totalEgressBandwidthTier:TIER_1

If the output shows the value DEFAULT, then Tier_1 networking isn't enabled.

REST

Call the Compute Engine APIinstances.get methodto view the network configuration.

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/RESOURCE_ID/

Replace the following:

The output should contain the following lines:

{ "name": RESOURCE_ID, "description": string, ... "networkPerformanceConfig": { "totalEgressBandwidthTier": "TIER_1" }, ... }

If the output shows the value DEFAULT, then Tier_1 networking is not enabled.

Create an instance template with Tier_1 networking

Use the Google Cloud console, the Google Cloud CLI or REST to create an instance template with per VM Tier_1 networking performance. Refer to the Creating an instance templatedocumentation to verify that you are meeting all the requirements to create your VM instance template.

Console

  1. In the Google Cloud console, go to the Instance templates page.
    Go to Instance templates
  2. Click Create instance template.
  3. Enter values for the following fields, or accept the default values.
  4. Specify a Name for your instance template. For more information, seeResource naming convention.
  5. Select a region and zone that supports Tier_1 networking.
  6. Select a Machine configuration for your VM. To create a VM with Tier_1 networking, you must select a supported machine series and machine type.
    • Click the General purpose tab before selecting N2, N2D, C4, C4A, C4D, C3, or C3D from the Series drop-down menu.
    • Click the Compute optimized tab before selecting C2 or C2D from the Series menu.
    • Click the Storage optimized tab before selecting Z3 from the Series menu.
  7. In the Machine type menu, choose a machine type that aligns with thebandwidth tier size requirements.
  8. In the Boot disk section, click Change, and then select a gVNIC-compatible orcustom image.
  9. Optional. In the Firewall section, choose your firewall rules.
  10. Expand the Advanced options section.
  11. Expand the Networking section, and then do the following:
  12. In the Network interface card menu, select gVNIC.
  13. In the Network bandwidth section, select theEnable per VM Tier_1 networking performance checkbox.
  14. If your VM has multiple NICs or you use IPv6 addresses, configure your Network interfaces.
  15. Click Create.

gcloud

Use the gcloud compute instance-templates create commandwith both the --network-performance-configs and the --network-interfaceflags.

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME
--image=OS_IMAGE
--machine-type=MACHINE_TYPE
--network-performance-configs=total-egress-bandwidth-tier=TIER_1
--network-interface=nic-type=GVNIC

Replace the following:

For example:

gcloud compute instance-templates create instance-template-1
--image-family=rocky-linux-8-optimized-gcp
--image-project=rocky-linux-cloud
--network-performance-configs=total-egress-bandwidth-tier=TIER_1
--machine-type=n2-standard-32
--network-interface=nic-type=GVNIC

REST

Call the Compute Engine APIinstanceTemplates.insert method. Within the request body, set the networkPerformanceConfig parameter to totalEgressBandwidthTier and TIER_1. Set the networkInterfacesparameter to nicType and GVNIC.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instancesTemplates

{ "name": "INSTANCE_TEMPLATE_NAME", "properties": { "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE", ... "networkPerformanceConfig": { "totalEgressBandwidthTier": "TIER_1" }, "networkInterfaces": [ { "nicType": "GVNIC" }, ... }

Replace the following:

Benchmark a higher bandwidth configuration

You canrun a benchmark testto check your VM's performance with per VM Tier_1 networking performance. Be sure to remove the benchmarking resourcesyou created during testing to avoid unexpected resource charges.

What's next