VPC-native clusters (original) (raw)
This page provides a general overview of VPC-native clusters in Google Kubernetes Engine (GKE).
This page is for Cloud architects and Networking specialists who design and architect the network for their organization. To learn more about common roles and example tasks that we reference in Google Cloud content, seeCommon GKE Enterprise user roles and tasks.
Overview
In GKE, clusters can be distinguished according to the way they route traffic from one Pod to another Pod.
A cluster that usesalias IP address ranges is called a_VPC-native cluster_. A cluster that uses custom static routes in a VPC network is called a routes-based cluster.
Best practice:
Plan and design your cluster configuration with your organization's Network architects, Network administrators, or any other Network engineers team responsible for defining, implementing, and maintaining your network architecture.
Benefits of VPC-native clusters
VPC-native clusters have several benefits:
- PodIP addresses are natively routable within the cluster's VPC network and other VPC networks connected to it by VPC Network Peering.
- Pod IP addresses are reserved in the VPC network before the Pods are created in your cluster. This prevents conflict with other resources in the VPC network and allows you to better plan IP address allocations.
- Pod IP address ranges do not depend on custom static routes. They do not consume thesystem-generated and custom static routes quota. Instead, automatically-generated subnet routes handle routing for VPC-native clusters.
- You can create firewall rules that apply to just Pod IP address ranges instead of any IP address on the cluster's nodes.
- Pod IP address ranges, and subnet secondary IP address ranges in general, are accessible from on-premises networks connected with Cloud VPN or Cloud Interconnect usingCloud Routers.
- Some features, such asnetwork endpoint groups (NEGs), only work with VPC-native clusters.
Default cluster network mode
VPC-native is the default network mode for all clusters in GKE versions 1.21.0-gke.1500 and later. For earlier versions, the default cluster network mode depends on how you create the cluster.
The following table lists the default cluster network mode for GKE cluster versions and cluster creation methods.
| GKE versions | Cluster creation method | Cluster network mode |
|---|---|---|
| All versions | The Google Cloud console | VPC-native |
| 1.21.0-gke.1500 and later | GKE API or Google Cloud CLI | VPC-native |
You can also create a routes-based cluster by specifying the--no-enable-ip-alias flag when you create the cluster.
IP address ranges for VPC-native clusters
When you create a VPC-native cluster, you specify a subnet in a VPC network. The cluster uses the following subnet IP address ranges:
IPv4 address allocation
VPC-native clusters allocate IPv4 addresses for nodes, Pods, and Services using distinct ranges within the specified subnet as follows.
- Node IP addresses: The cluster utilizes the subnet's primary IPv4 address range to assign IP addresses to all nodes.
- Pod IP addresses: The cluster utilizes the secondary IPv4 address range within the subnet for all Pod IPv4 addresses within the cluster. For enhanced flexibility, you can use additional subnet secondary IPv4 address ranges by configuring discontiguous multi-Pod CIDR.
- Service IP addresses: The cluster utilizes a separate secondary IP address range for all Service (cluster IP) addresses. For information on enabling multiple clusters to share the same Services IPv4 range, refer toSharing IP address ranges across GKE clusters.
IPv6 address allocation (dual-stack networking)
- Node and Pod IPv6 addresses: In clusters enabled for dual-stack networking, the node's IPv6 address and all Pod IPv6 addresses originate from the node's designated
/96IPv6 address range. The node IP address itself is the first/128(single IPv6 address) within this range. For more information, see dual-stack networking.
The following table provides a summary of IP address ranges for nodes, Pods, and Services:
| Range | Explanation | Example |
|---|---|---|
| Nodes | Node IP addresses are assigned from theprimary IP address range of the subnet associated with your cluster. Both node IP addresses and the size of the subnet's secondary IP address range for Pods limit the number of nodes that a cluster can support. Refer tonode limiting ranges for more information. | If you plan to create a 900-node cluster, the primary IP address range of the cluster's subnet must be at least a /22 (2(32-22) = 210 = 1,024 addresses). Of those 1,024 addresses, 1,020 are usable becausefour IP addresses are reserved in every primary IP address range. Refer to Subnet primary IP address range andSubnet secondary IP address range for Pods for more information. |
| Pods | Pod IP addresses are taken from the cluster subnet's secondary IP address range for Pods. Unless you set a different maximum number of Pods per node, GKE allocates a /24 alias IP range (256 addresses) to each node for the Pods running on it. On each node, those 256 alias IP addresses are used to support up to 110 Pods. | For a 900-node cluster supporting up to 110 Pods per node, you need 900 × 256 = 230,400 IP addresses for Pods. (Each node is allocated an alias IP range whose netmask's size is /24.) This cluster requires a subnet whose secondary IP range for Pods has a subnet mask no larger than /14. This secondary IP range provides 2(32-14) = 218 = 262,144 IP addresses for Pods. Refer to Subnet secondary IP address range for Pods for more information. |
| Services | Service (cluster IP) addresses are taken from the cluster's subnet's secondary IP address range for Services. You must ensure this range is large enough to provide addresses for all the Kubernetes Services you host in your cluster. In GKE Autopilot clusters running version 1.27 and later, and GKE Standard clusters running version 1.29 and later, GKE assigns IP addresses for GKE Services from a GKE-managed range: 34.118.224.0/20 by default. This eliminates the need for you to specify your own IP address range for Services. For details, seeSubnet secondary IP address range for Services. | For a cluster that runs up to 3000 Services, you need 3000 cluster IP addresses. You need a secondary range of size /20 or larger. A /20 range of IP addresses results in 2(32-20) = 212 = 4,096 IP addresses. Refer to Subnet secondary IP address range for Services for more information. |
Internal IP addresses
The IP addresses you use for your VPC-native cluster's subnets must come from a valid subnet range. The valid ranges include private IP addresses (RFC 1918 and others) and privately used public IP addresses. SeeValid ranges andRestricted ranges in the VPC documentation for more information about valid subnet ranges.
SeeUsing non-RFC 1918 private IP address rangesfor instructions on enabling the use of these ranges.
SeeEnable privately used public IP address rangesfor instructions on the use of these ranges.
Secondary range assignment methods
You can assign Pod IP address ranges and Service (ClusterIP) address ranges to a VPC-native cluster. These IP address ranges can be managed by GKE or user-managed.
You must understand the following key terms to understand the secondary range assignment methods.
Assignment: assigning IP address ranges refers to the process of allocating a specific subnet range to a VPC-native cluster. This establishes a pool of IP addresses that the components can use within the cluster, such as Pods and Services.
Management: Managing the IP address range refers to the ongoing (creation, update, deletion, reading) CRUD operations at the cluster, node pool, or Pod level, related to the assigned subnet ranges and resource allocation within your VPC-native cluster.
GKE-managed secondary ranges (default)
For GKE Autopilot clusters running version 1.27 and later and GKE Standard clusters running versions 1.29 and later, GKE assigns IP addresses for Services from a GKE-managed range by default: 34.118.224.0/20. This eliminates the need for you to specify your own IP address range for Services. The following considerations apply:
- You can optionally specify custom ranges for Services by using the
--services-ipv4-cidrflag. - If you specify only a range size using the
--services-ipv4-cidrflag (for example,/22), GKE still uses the GKE-managed range to obtain the sub-range of addresses. - GKE doesn't create a separate secondary IP address range for Services when the GKE-managed range is used.
User-managed
For full control over IP address allocation, you can manually manage your VPC-native cluster's subnets.
You can create the subnet's secondary IP address ranges, then create a cluster that uses those ranges. During cluster creation, specify the subnet range name for Pods and Services. If you manually create the secondary ranges, you must manage them yourself.
The smallest IP address range you can create without using discontiguous multi-Pod CIDR is /28, but that range would only let you create 1 node with a maximum of 8 Pods. You should use a range that is large enough for the maximum number of nodes that you need.
The minimum usable range also depends on themaximum number of Pods per Node.
Refer to the table inOptimizing IP address allocationfor the minimum usable CIDR range for different values of maximum Pods per node.
If you exhaust your IP address range for Pods, you must do one of the following:
- Create a new cluster with a larger Pod address range.
- Re-create your node pools after decreasing the--max-pods-per-nodefor the node pools.
- Expand the secondary Pod IP address range using discontiguous multi-Pod CIDR.
Differences with routes-based clusters
The allocation scheme for Pod and Service (ClusterIP) addresses is different than the scheme used by a routes-based cluster. Instead of specifying a single CIDR for Pods and Services together, you must choose or create two secondary IP address ranges in cluster's subnet: one for Pods and another for Services.
Shared VPC considerations
When creating a VPC-native cluster in aShared VPCenvironment, a project owner, editor, or Identity and Access Management (IAM) principal with the Network Admin role in the Shared VPC host project must create the cluster's subnet and secondary IP address ranges manually. A service project admin who creates a cluster must at least havesubnet-level permissions to the subnet in the Shared VPC network's host project.
In a Shared VPC environment, secondary IP address ranges cannot be managed by GKE. A Network Admin in the Shared VPC host project must create the subnet and secondary IP address ranges before you can create the cluster. For an example showing how to set up a VPC-native cluster in a Shared VPC network, refer toSetting up clusters with Shared VPC.
IP address range planning
Use the information in the following sections to help you calculate sizes for primary and secondary IP address ranges of the subnet used by your cluster.
Subnet primary IP address range
Consider the following conditions when planning your primary node IP address range:
- Every subnet must have a primary IP address range. This is the IP address range that GKE uses to allocate IP addresses for internal load balancers and nodes.
- You cannot shrink or change a subnet's primary IP address range after the subnet has been created.
- The first two and last two IP addresses of a primary IP address range are reserved by Google Cloud.
- Clusters with Private Service Connect use the primary subnet range to provision the internal IP address assigned to the control plane endpoint. However, you can override this IP address provisioning with the private-endpoint-subnetwork flag. To learn more, see Create a cluster and select the control plane IP address range.
The following table shows the maximum number of nodes you can create given the size of the subnet's primary IP address range and the cluster configuration:
- Scenario 1: Maximum nodes in a cluster that uses the default subnet.
- Scenario 2: Maximum nodes in a Private Service Connect cluster that doesn't use the
private-endpoint-subnetworkflag.
| Subnet primary IP range | Scenario 1 | Scenario 2 |
|---|---|---|
| /29 Minimum size for a subnet's primary IP range | 4 nodes | 3 nodes |
| /28 | 12 nodes | 11 nodes |
| /27 | 28 nodes | 27 nodes |
| /26 | 60 nodes | 59 nodes |
| /25 | 124 nodes | 123 nodes |
| /24 | 252 nodes | 251 nodes |
| /23 | 508 nodes | 507 nodes |
| /22 | 1,020 nodes | 1,019 nodes |
| /21 | 2,044 nodes | 2,043 nodes |
| /20 Default size of a subnet's primary IP range inauto mode networks | 4,092 nodes | 4,091 nodes |
| /19 | 8,188 nodes | 8,187 nodes |
| /8 Maximum size for a subnet's primary IP range | 16,777,212 nodes | 16,777,211 nodes |
Expand the primary IP address range
If you run out of IP addresses in the primary IP address range, you canexpand the primary IP address rangeat any time, even when Google Cloud resources, such as load balancers and network endpoint groups, use the subnet.
Before you expand the primary IP address range, consider the following:
- There must be no overlapping IP address ranges in the subnet.
- GKE uses the primary IP address range to allocate IP addresses for internal load balancers and nodes.
Useful formulas
You can use the following formulas to:
- Calculate the maximum number of nodes, N, that a given netmask can support in clusters that use the default subnet. Use S for the size of the netmask, whose valid range is between
8and29, inclusive.
N = 2(32 -S) - 4 - Calculate the size of the netmask, S, required to support a maximum of N nodes:
S = 32 - ⌈log2(N + 4)⌉⌈⌉is the ceiling (least integer) function, meaning round up to the next integer. The valid range for the size of the netmask, S, is between8and29, inclusive.
In Private Service Connect clusters that don't use theprivate-endpoint-subnetwork flag, you can use the preceding formulas, but reduce the value of N by 1.
Subnet secondary IP address range for Pods
Carefully plan your secondary IP address range for Pods.
The following table shows the maximum number of nodes and Pods you can create in all clusters that use the subnet, given the size of the subnet's secondary IP address range used by Pods.
This table assumes themaximum number of Pods per nodeis 110, which is the default Pod density.
| Subnet secondary IP range for Pods | Maximum Pod IP addresses | Maximum nodes | Maximum Pods |
|---|---|---|---|
| /24 Smallest possible Pod IP rangewhen the secondary range assignment method is user-managed | 256 addresses | 1 node | Autopilot: 32 Pods Standard: 110 Pods |
| /23 Only possiblewhen the secondary range assignment method is user-managed | 512 addresses | 2 nodes | Autopilot: 64 Pods Standard: 220 Pods |
| /22 Only possiblewhen the secondary range assignment method is user-managed | 1,024 addresses | 4 nodes | Autopilot: 128 Pods Standard: 440 Pods |
| /21 Smallest possible Pod IP rangewhen the secondary range assignment method is managed by GKE | 2,048 addresses | 8 nodes | Autopilot: 256 Pods Standard: 880 Pods |
| /20 | 4,096 addresses | 16 nodes | Autopilot: 512 Pods Standard: 1,760 Pods |
| /19 | 8,192 addresses | 32 nodes | Autopilot: 1,024 Pods Standard: 3,520 Pods |
| /18 | 16,384 addresses | 64 nodes | Autopilot: 2,048 Pods Standard: 7,040 Pods |
| /17 | 32,768 addresses | 128 nodes | Autopilot: 4,096 Pods Standard: 14,080 Pods |
| /16 | 65,536 addresses | 256 nodes | Autopilot: 8,192 Pods Standard: 28,160 Pods |
| /15 | 131,072 addresses | 512 nodes | Autopilot: 16,384 Pods Standard: 56,320 Pods |
| /14 Default size for the subnet's secondary IP address range for Podswhen the secondary range assignment method is managed by GKE | 262,144 addresses | 1,024 nodes | Autopilot: 32,768 Pods Standard: 112,640 Pods |
| /13 | 524,288 addresses | 2,048 nodes | Autopilot: 65,536 Pods Standard: 225,280 Pods |
| /12 | 1,048,576 addresses | 4,096 nodes | Autopilot: 131,072 Pods Standard: 450,560 Pods |
| /11 | 2,097,152 addresses | 8,192 nodes | Autopilot: 262,144 Pods Standard: 901,120 Pods |
| /10 | 4,194,304 addresses | 16,384 nodes | Autopilot: 524,288 Pods Standard: 1,802,240 Pods |
| /9 Largest possible Pod address range | 8,388,608 addresses | 32,768 nodes | Autopilot: 1,048,576 Pods Standard: 3,604,480 Pods |
If you have changed the maximum number of Pods per node, you can use the following formulas to calculate the maximum number of nodes and Pods that a subnet's secondary IP address range for Pods can support:
- Calculate the size of the netmask of each node's Pod range, M.
M = 31 - ⌈log2(Q)⌉where:- Q is the number of Pods per node
⌈⌉is the ceiling (least integer) function, meaning round up to the next integer- For example, if Q is 110, then
M = 24
- Calculate the maximum number of nodes, N, that the subnet's secondary IP address range for Pods can support:
N = 2(M - S)where:- M is the size of the netmask of each node's alias IP address range for Pods, calculated in the first step
- S is the size of the subnet mask of the subnet's secondary IP address range
- For example, if M is 24, and S is 20, then
N = 16
- Calculate the maximum number of Pods, P, that the subnet's secondary IP address range for Pods can support:
P = N × Qwhere:- N is the maximum number of nodes, calculated in the previous step
- Q is the number of Pods per node
- For example, if N is 16, and Q is 110, then
P = 1760
You can add more IP addresses for Pods by using discontiguous multi-Pod CIDR.
Subnet secondary IP address range for Services
Carefully plan your secondary IP address range for Services. Because this is also a subnet secondary IP address range, this range cannot be changed once the cluster is created.
If you usemulti-cluster services, the ServiceImport object uses IP addresses from the secondary IP address range for Services.
In GKE Autopilot clusters running version 1.27 and later and GKE Standard clusters running versions 1.29 and later, GKE assigns IP addresses for Services from a GKE-managed range by default: 34.118.224.0/20. This eliminates the need for you to specify your own IP address range for Services. The following considerations apply:
- You can optionally specify custom ranges for Services by using the
--services-ipv4-cidrflag or the--services-secondary-range-nameflag. - If you specify only a range size using the
--services-ipv4-cidrflag (for example,/22), GKE still uses the GKE-managed range to obtain the sub-range of addresses. - GKE doesn't create a separate secondary IP address range for Services when the Google-managed range is used. The GKE-managed range doesn't use the secondary IP address range quota for your subnet.
The following table shows the maximum number of Services you can create in a single cluster using the subnet, given the size of the subnet's secondary IP address range for Services.
| Secondary IP range for Services | Maximum number of Services |
|---|---|
| /28 Smallest possible Service address rangewhen the secondary range assignment method is user-managed | 16 Services |
| /27 Smallest possible Service address rangewhen the secondary range assignment method is managed by GKE | 32 Services |
| /26 | 64 Services |
| /25 | 128 Services |
| /24 | 256 Services |
| /23 | 512 Services |
| /22 | 1,024 Services |
| /21 | 2,048 Services |
| /20 Default size for the subnet's secondary IP range for Serviceswhen the secondary range assignment method is managed by GKE | 4,096 Services |
| /19 | 8,192 Services |
| /18 | 16,384 Services |
| /17 | 32,768 Services |
| /16 Largest possible Service address range | 65,536 Services |
Sharing IP address ranges across GKE clusters
You can share the primary range, secondary IP address range for Pods, and secondary IP address range for Services between clusters in the same subnetwork. This behavior is available for both Standard and Autopilot clusters.
You might want to share IP address ranges if you have a centralized team that is managing the infrastructure for clusters. You can reduce overhead by creating three ranges, for Pods, Services and nodes, and reusing or sharing them, especially in a Shared VPC model. It can also make it easier for network administrators to manage IP addresses by not requiring them to create specific subnets for each cluster.
Sharing the customized subnet range for the control plane
By default, GKE uses the primary subnet range to provision the internal endpoint of the control plane. However, in clusters withPrivate Service Connect, you can configure GKE to provision the internal endpoint from a different subnet that you created. You can share this subnet among other clusters, or across projects if you are using Shared VPC.
Sharing the primary IP address range for nodes
If you create more than one cluster in the subnet, the primary IP address range for nodes is shared by default.
Sharing the primary IP address for nodes has the following limitations:
- If you share the primary IP address range for nodes with two or more VPC-native clusters, one cluster can use a large portion of the shared IP address range, leaving the other clusters without enough IP addresses to expand.
When you share the secondary range for Pods, each Pod still gets a unique IP address.
Sharing the secondary IP address range for Pods has the following limitations:
- If you share the secondary IP address range for Pods with two or more VPC-native clusters, one cluster can use a large portion of the shared IP address range, leaving the other clusters without enough IP addresses to expand.
- Among the different types of secondary ranges, both theGKE-managedand additional Pod rangesare not shareable across clusters.
- To share a secondary IP address range, pass it on the command line with
--cluster-secondary-range. You cannot use a shared secondary range when creating clusters in the UI.
Two or more clusters can simultaneously use the same subnet secondary IPv4 address range for Services when you use user-managed secondary ranges.
To configure two or more clusters to share a common subnet secondary IPv4 address range for Services, use the same subnet secondary IPv4 address range when you create each cluster. There is no separate configuration flag required to share a common IPv4 address range for Services.
When sharing a common IPv4 address range for Services, each cluster uses the entire subnet secondary IPv4 address range for Services internally. The IP addresses for Services are programmed within each cluster's node, but they are not assigned to the network interface of any node. Service IP addresses are not routable within the cluster's VPC network. Service IP addresses are only usable by client Pods within the same cluster as the Service.
When a Pod sends a packet to a Service IP address, the iptables or eBPF configuration on the node performs destination Network Address Translation (NAT), changing the destination IP address of the packet from the Service IP address to a serving Pod IP address. The packet is routed based on the destination Pod IP address.
Sharing the secondary IP address range for Services provides the following benefits:
- Reduce the number of unique secondary IP address ranges for Services created on a subnet
- Use fewer IP addresses
Sharing the secondary IP address range for Services has the following limitations:
- Sharing the secondary IP address range for Services is not supported withVPC scope Cloud DNS for GKE.
- You can't share ranges that match the following regex:
^gke-.*-services-[abcdef0-9]{8} - To share a secondary IP address range for services, pass it on the command line with
--cluster-secondary-rangeyou cannot use a shared secondary range for services when creating clusters in the UI.
Node limiting ranges
The maximum number of Pods and Services for a given GKE cluster is limited by the size of the cluster's secondary ranges. The maximum number of nodes in the cluster is limited by the size of the cluster's subnet's primary IP address range and the cluster's Pod address range.
The following error message indicates that either the subnet's primary IP address range or the cluster's Pod IP address range (the subnet's secondary IP address range for Pods) has been exhausted:
Instance [node name] creation failed: IP space of [cluster subnet] is
exhausted
You can add more IP addresses for nodes byexpanding the primary subnet, or add new IP addresses for Pods usingdiscontiguous multi-Pod CIDR. For more information, seeNot enough free IP address space for Pods.
IPv4/IPv6 dual-stack networking
With IPv4/IPv6 dual-stack networking, you can define how GKE allocates IP addresses (ipFamilies) to the following objects:
- For Pods and nodes, GKE allocates both IPv4 and IPv6 addresses.
- For Services, GKE allocates either single-stack (IPv4 only or IPv6 only), or dual-stack addresses.
In GKE version 1.24 or later, you can enable dual-stack networking for new GKE clusters on standalone and Shared VPC networks. You can also apply network policies with dual-stack networking enabled. If you see validation errors when enabling dual-stack networking on GKE clusters that have been upgraded from versions 1.24 to versions 1.25 or 1.26, contact the Google Cloud support team.
Benefits
Dual-stack networking provides the following benefits:
- Enables end-to-end IPv6 communication.
- Enables better performance compared to network address translation (NAT) or IP tunneling. This is achieved because there is no IPv6 to IPv4 translation.
Availability
Dual-stack networking with GKE has the following restrictions:
- Dual-stack networking is only available for VPC-native clusters clusters withGKE Dataplane V2 enabled.
- Dual-stack networking is only supported on subnets in custom mode VPCs. For more information, see Google Cloud types of VPC networks.
- Single-stack IPv6 addresses for Pods or nodes are not supported.
- Dual-stack clusters consume additional memory per node to support both IPv4 and IPv6 compared to IPv4 only clusters. Consider this characteristic when setting up large-scale clusters.
- Dual-stack clusters don't support Private Google Access over IPv6.
- GKE versions 1.26.0-gke.2200 and later supports IPv6 (AAAA records) with Cloud DNS for cluster-internal operations and external DNS queries.
- Dual-stack Services are supported for
ClusterIP,NodePort, andLoadBalancerServices. - IPv6 is not supported for Windows nodes.
Consider the preceding restrictions before creating a cluster with dual-stack networking. For more information, learnhow to create a VPC-native cluster with dual-stack networking.
Public and Private IPv6 address assignment
The following table provides a summary of public and private IPv6 addresses with dual-stack networking behavior and configurations:
| ipv6-access-type flag | IP address assignment | Subnet range |
|---|---|---|
| EXTERNAL | GKE assigns external IPv6 addresses that are routable to the internet. | From 2600:1900/28 |
| INTERNAL | GKE assigns internal IPv6 addresses that are not routable to the internet. Clusters with INTERNAL IPv6 access type cannot access the internet over IPv6 addresses. Cloud NAT does not support IPv6 addresses. | From fd20::/20 (which is a subset of the overall ULA range: fc00::/7). |
To learn more, see how to use a dual-stack network for a VPC-native cluster.
Architecture
A cluster with IPv4/IPv6 dual-stack networking has the following ranges allocated:
- A /64 range to each subnet as a primary range.
- A /96 per-node range from the primary range to use as a primary node IP address range.
- A /112 per-node range from the primary node IP address range to use as a Pod IP address range for that node. With dual-stack networking, Pods get their IPv6 addresses from the overall Pod IP address range(similar to nodes), and not from secondary range for Pods reserved to IPv4 addresses.
The overall pod IP address range is comprised of non-overlapping ranges from the primary node IP range. Therefore, this Pod IP range is discontiguous. - A /112 range to use for Services. This range comes from a /64 range from the Google private address range which has been reserved for GKE's secondary Services IP address range.
The following diagram shows how Google Cloud and GKE allocate IPv6 addresses:
In the diagram, the primary range of the VPC subnet is2600:1900:0:1::/64 and the reserved range for GKE Services is2600:2D00:0:4::0:0/64. Each node in the cluster has a /96 range for the primary node IP address range and a /112 range for the Pod IP address range. The cluster also has a /112 secondary Services IP address range.
Services
You can create an IPv4/IPv6 dual-stack Service of typeClusterIPorNodePort. New GKE clusters running version 1.29 or later support dual-stack Services of typeLoadBalancer.
You can expose a Deployment with a Service of type ClusterIP, NodePort, orLoadBalancer. For each of these Service types, you can define ipFamilies andipFamilyPolicy fields as either IPv4, IPv6, or adual-stackService. For more information, see anexample of how to set up a Deployment.