Internal passthrough Network Load Balancer overview (original) (raw)

An internal passthrough Network Load Balancer is a regional load balancer that is built on the Andromeda network virtualization stack.

Internal passthrough Network Load Balancers distribute traffic among internal virtual machine (VM) instances in the same region in a Virtual Private Cloud (VPC) network. It enables you to run and scale your services behind an internal IP address that is accessible only to systems in the same VPC network or systems connected to your VPC network.

Use an internal passthrough Network Load Balancer in the following circumstances:

The internal passthrough Network Load Balancers address many use cases. For a few high-level examples, seePassthrough Network Load Balancer overview.

How internal passthrough Network Load Balancers work

An internal passthrough Network Load Balancer has a frontend (the forwarding rule) and a backend (the backend service). You can use either instance groups or GCE_VM_IP zonal NEGs as backends on the backend service. This example shows instance group backends.

High-level internal passthrough Network Load Balancer example.

High-level internal passthrough Network Load Balancer example (click to enlarge).

Unlike a proxy load balancer, an internal passthrough Network Load Balancer doesn't terminate connections from clients and then open new connections to backends. Instead, an internal passthrough Network Load Balancer routes connections directly from clients to eligible backends, without a proxy between the clients and backends. Responses from each selected backend are delivered using direct server return. For more information, seeTraffic distribution for internal passthrough Network Load Balancersand IP addresses for request and return packets.

The load balancer monitors backend health by using health check probes. For more information, see the Health check section.

The Google Cloud Linux guest environment, Windows guest environment, or an equivalent process configures each backend VM with the IP address of the load balancer. For VMs created from Google Cloud images, the Guest agent (formerly, the Windows Guest Environment or Linux Guest Environment) installs the local route for the load balancer's IP address. Google Kubernetes Engine instances based on Container- Optimized OS implement this by using iptablesinstead.

Google Cloud virtual networking manages traffic delivery and scaling as appropriate.

Protocols, scheme, and scope

Each internal passthrough Network Load Balancer supports the following:

An internal passthrough Network Load Balancer doesn't support the following:

Client access

By default, the load balancer only supports clients that are in the same region as the load balancer. Clients can be in the same network as the load balancer or in a VPC network connected by usingVPC Network Peering. You can enable global access to allow clients from any region to access your internal passthrough Network Load Balancer.

Internal passthrough Network Load Balancer with global access.

Internal passthrough Network Load Balancer with global access (click to enlarge).

The following table summarizes client access.

Global access disabled Global access enabled
Clients must be in the same region as the load balancer. They also must be in the same VPC network as the load balancer or in a VPC network that is connected to the load balancer's VPC network by using VPC Network Peering. Clients can be in any region. They still must be in the same VPC network as the load balancer or in a VPC network that's connected to the load balancer's VPC network by using VPC Network Peering.
On-premises clients can access the load balancer throughCloud VPN tunnels or VLAN attachments. These tunnels or attachments must be in the same region as the load balancer. On-premises clients can access the load balancer through Cloud VPN tunnels or VLAN attachments. These tunnels or attachments can be in any region.

IP addresses for request and return packets

When a backend VM receives a load-balanced packet from a client, the packet's source and destination are as follows:

Because the load balancer is a pass-through load balancer (not a proxy), packets arrive bearing the destination IP address of the load balancer's forwarding rule. Configure software running on backend VMs to do the following:

Return packets are sent directly from the load balancer's backend VMs to the client. The return packet's source and destination IP addresses depend on the protocol:

The following table summarizes sources and destinations for response packets:

Traffic type Source Destination
TCP The IP address of the load balancer's forwarding rule The requesting packet's source
UDP For most use cases, the IP address of the load balancer's forwarding rule 1 The requesting packet's source

1 It is possible to set the response packet's source to the VM NIC's primary internal IPv4 address or an alias IP address range. If the VM has IP forwarding enabled, arbitrary IP address sources can also be used. Not using the forwarding rule's IP address as a source is an advanced scenario because the client receives a response packet from an internal IP address that doesn't match the IP address to which it sent a request packet.

Architecture

An internal passthrough Network Load Balancer with multiple backends distributes connections among all of those backends. For information about the distribution method and its configuration options, see traffic distribution.

You can use either instance groups or zonal NEGs, but not a combination of both, as backends for an internal passthrough Network Load Balancer:

High availability describes how to design an internal load balancer that isn't dependent on a single zone.

Instances that participate as backend VMs for internal passthrough Network Load Balancers must be running the appropriate Linux or Windows guest environment or other processes that provide equivalent functionality. This guest environment must be able to contact the metadata server (metadata.google.internal, 169.254.169.254) to read instance metadata so that it can generate local routes to accept traffic sent to the load balancer's internal IP address.

This diagram shows traffic distribution among VMs located in two separate instance groups. Traffic sent from the client instance to the IP address of the load balancer (10.10.10.9) is distributed among backend VMs in either instance group. Responses sent from any of the serving backend VMs are delivered directly to the client VM.

You can use an internal passthrough Network Load Balancer with either acustom mode or auto mode VPC network. You can also create internal passthrough Network Load Balancers with an existinglegacy network.

An internal passthrough Network Load Balancer doesn't support the following:

Internal IP address

Internal passthrough Network Load Balancers support IPv4-only, dual-stack, and IPv6-only subnets. For more information about each one, see Types of subnets.

An internal passthrough Network Load Balancer requires at least one forwarding rule. The forwarding rule references the internal IP address:

Firewall configuration

Internal passthrough Network Load Balancers require the following configuration for hierarchical firewall policies and VPC firewall rules:

For more information, see Configuring firewall rules.

Forwarding rules

A forwarding rule specifies the protocol and ports on which the load balancer accepts traffic. Because internal passthrough Network Load Balancers aren't proxies, they pass traffic to backends on the same protocol and port.

An internal passthrough Network Load Balancer requires at least one internal forwarding rule. You can define multiple forwarding rules for the same load balancer.

If you want the load balancer to handle both IPv4 and IPv6 traffic, create two forwarding rules: one rule for IPv4 traffic that points to IPv4 (or dual-stack) backends, and one rule for IPv6 traffic that points only to dual-stack backends. It's possible to have an IPv4 and an IPv6 forwarding rule reference the same backend service, but the backend service must reference dual-stack backends.

The forwarding rule must reference a specificsubnet in the same VPC network and region as the load balancer's backend components. This requirement has the following implications:

Forwarding rule protocols

Internal passthrough Network Load Balancers support the following IPv4 protocol options for each forwarding rule: TCP, UDP, or L3_DEFAULT.

Internal passthrough Network Load Balancers support the following IPv6 protocol options for each forwarding rule: TCP or UDP.

The L3_DEFAULT option enables you to load balance TCP, UDP, ICMP, ICMPv6, SCTP, ESP, AH, and GRE protocols.

In addition to supporting protocols other than TCP and UDP, theL3_DEFAULT option makes it possible for a single forwarding rule to simultaneously forward traffic for multiple protocols. For example, in addition to making HTTP requests, you can also ping the load balancer IP address.

Forwarding rules that use the TCP or UDP protocols can reference a backend service by using either the same protocol as the forwarding rule or a backend service using the UNSPECIFIED protocol.

If you're using the L3_DEFAULTprotocol, you must configure the forwarding rule to accept traffic on all ports. To configure all ports, either set--ports=ALL by using the Google Cloud CLI, or set allPorts toTrue by using the API.

The following table summarizes how to use these settings for different protocols:

Traffic to be load-balanced Forwarding rule protocol Backend service protocol
TCP (IPv4 or IPv6) TCP TCP or UNSPECIFIED
UDP (IPv4 or IPv6) UDP UDP or UNSPECIFIED
TCP, UDP, ICMP, ICMPv6, SCTP, ESP, AH, and GRE L3_DEFAULT UNSPECIFIED

Forwarding rules and global access

An internal passthrough Network Load Balancer's forwarding rules are regional, even when global access is enabled. After you enable global access, the regional internal forwarding rule's allowGlobalAccess flag is set to true.

Forwarding rules and port specifications

When you create an internal forwarding rule, you must choose one of the following port specifications:

An internal forwarding rule that supports either all TCP ports or all UDP ports allows backend VMs to run multiple applications, each on its own port. Traffic sent to a given port is delivered to the corresponding application, and all applications use the same IP address.

When you need to forward traffic on more than five specific ports, combinefirewall rules with forwarding rules. When you create the forwarding rule, specify all ports, and then create ingress allow firewall rules that only permit traffic to the desired ports. Apply the firewall rules to the backend VMs.

You cannot modify a forwarding rule after you create it. If you need to change the specified ports or the internal IP address for an internal forwarding rule, you must delete and recreate it.

Multiple forwarding rules for a single backend service

You can configure multiple internal forwarding rules that all reference the same internal backend service. An internal passthrough Network Load Balancer requires at least one internal forwarding rule.

Configuring multiple forwarding rules for the same backend service lets you do the following:

For more information about scenarios involving two or more internal forwarding rules that share a common internal IP address, see Multiple forwarding rules with the same IP address.

When using multiple internal forwarding rules, make sure that you configure the software running on your backend VMs to bind to all of the forwarding rule IP addresses or to any address (0.0.0.0/0 for IPv4 or ::/0 for IPv6). The destination IP address for a packet delivered through the load balancer is the internal IP address associated with the corresponding internal forwarding rule. For more information, see TCP and UDP request and return packets.

Backend service

Each internal passthrough Network Load Balancer has one regional internal backend service that defines backend parameters and behavior. The name of the backend service is the name of the internal passthrough Network Load Balancer shown in the Google Cloud console.

Each backend service defines the following backend parameters:

Each backend service operates in a single region and distributes traffic for backend VMs in a single VPC network:

Instance group backends and network interfaces

Within a given (managed or unmanaged) instance group, the nic0 network interface of each member VM is always in the same VPC network:

Member VMs can have additional network interfaces (vNICs or Dynamic Network Interfaces). Each non-nic0 interface can be in either the instance group's VPC network (the network used by the nic0 interface) or a different VPC network.

To distribute load-balanced traffic to a non-nic0 network interface, we recommend you use zonal NEGs with GCE_VM_IP endpoints. However, it is possible to set the VPC network of a backend service to match a VPC network that contains non-nic0 network interfaces of VMs in an instance group. For more information, seeBackend service network specificationand Backend service network rules.

Zonal NEG backends and network interfaces

When you create a new zonal NEG with GCE_VM_IP endpoints, you must explicitly associate the NEG with a subnetwork of a VPC network before you can add any endpoints to the NEG. Neither the subnet nor the VPC network can be changed after the NEG is created.

Within a given NEG, each GCE_VM_IP endpoint actually represents a network interface. The network interface must be in the subnetwork associated with the NEG. From the perspective of a Compute Engine instance, the network interface can use anyidentifier. From the perspective of being an endpoint in a NEG, the network interface is identified by using its primary internal IPv4 address. For more information, see NEGs with GCE_VM_IP endpoints.

There are two ways to add a GCE_VM_IP endpoint to a NEG:

Backend service network specification

You can explicitly associate a VPC network with a backend service by using the --network flag when you create the backend service. Thebackend service network rules go into effect immediately.

If you omit the --network flag when you create a backend service, Google Cloud uses one of the following qualifying events to implicitly set the backend service's associated VPC network. After it is set, the backend service's associated VPC network cannot be changed:

After the backend service's VPC network has been set by a qualifying event, any additional forwarding rules or backend groups that you add must follow the backend service network rules.

Backend service network rules

The following rules apply after a backend service is associated with a specific VPC network:

Dual-stack backends (IPv4 and IPv6)

If you want the load balancer to use dual-stack backends that handle both IPv4 and IPv6 traffic, note the following requirements:

IPv6-only backends

If you want the load balancer to use IPv6-only backends, note the following requirements:

Note that IPv6-only VMs can be created under both dual-stack and IPv6-only subnets, but dual-stack VMs can't be created under IPv6-only subnets.

Backend subsetting

Backend subsetting is an optional feature that improves performance by limiting the number of backends to which traffic is distributed.

We recommend that you only enable subsetting if you need to support more than 250 backend VMs on a single load balancer. For more information, seeBackend subsetting for internal passthrough Network Load Balancer.

Health check

Health check information is used to determine eligible backends for new connections, and you can control whether existing connections persist on unhealthy backends. For more information about eligible backends, see Traffic distribution for internal passthrough Network Load Balancers.

Health check type, protocol, and port

The load balancer's backend service must reference a global or regional health check, using any supported health check protocol and port. The health check protocol and port details don't have to match the load balancer backend service protocol and forwarding rule IP port information.

Because all supported health check protocols rely on TCP, when you use an internal passthrough Network Load Balancer to balance connections and traffic for other protocols, backend VMs must run a TCP-based server to answer health check probers. For example, you can use an HTTP health check combined with running an HTTP server on each backend VM. In this example, your scripts or software are responsible for configuring the HTTP server so that it returns status 200 only when the software listening to load-balanced connections is operational.

For more information about supported health check protocols and ports, seeHealth check categories, protocols, and portsand How health checks work.

Health check packets

Health check probers send packets to the backend VM network interface that's in the VPC network that matches the Backend service network specification. Health check packets have the following characteristics:

Software running on the backend VMs must bind to and listen on relevant IP address and port combinations. The simplest way to accomplish this is to configure software to bind to and listen on the relevant ports of any of the VM's IP addresses (0.0.0.0). For more information, see Destination for probe packets.

High availability architecture

The internal passthrough Network Load Balancer is highly available by design. There are no special steps to make the load balancer highly available because the mechanism doesn't rely on a single device or VM instance.

To deploy your backend VM instances to multiple zones, follow these deployment recommendations:

The following table summarizes the component requirements for internal passthrough Network Load Balancers used with Shared VPC networks. For an example, see creating an internal passthrough Network Load Balancer on the Provisioning Shared VPC page.

IP address Forwarding rule Backend components
An internal IP address must be defined in the same project as the backend VMs.For the load balancer to be available in a Shared VPC network, the Google Cloud internal IP address must be defined in the same service project where the backend VMs are located, and it must reference a subnet in the desired Shared VPC network in the host project. The address itself comes from the primary IP range of the referenced subnet.If you create an internal IP address in a service project and the IP address subnet is in the service project's VPC network, your internal passthrough Network Load Balancer is local to the service project. Your internal passthrough Network Load Balancer isn't local to any Shared VPC host project. An internal forwarding rule must be defined in the same project as the backend VMs.For the load balancer to be available in a Shared VPC network, the internal forwarding rule must be defined in the same service project where the backend VMs are located, and it must reference the same subnet (in the Shared VPC network) that the associated internal IP address references.If you create an internal forwarding rule in a service project and the forwarding rule's subnet is in the service project's VPC network, your internal passthrough Network Load Balancer is local to the service project. Your internal passthrough Network Load Balancer isn't local to any Shared VPC host project. In a Shared VPC scenario, backend VMs are located in a service project. A regional internal backend service and health check must be defined in that service project.

Traffic distribution

Internal passthrough Network Load Balancers support a variety of traffic distribution customization options, including session affinity, connection tracking, and failover. For details about how internal passthrough Network Load Balancers distribute traffic, and how these options interact with each other, see Traffic distribution for internal passthrough Network Load Balancers.

Quotas and limits

For information about quotas and limits, seeLoad balancing resource quotas.

Limitations

What's next