Maximum transmission unit (original) (raw)

The maximum transmission unit (MTU)is the size, in bytes, of the largest possible IP packet, including IP headers, Layer 4 protocol headers, and Layer 4 data, that can fit inside an Ethernet frame.

Valid VPC network MTU sizes

Virtual Private Cloud (VPC) networks use a default MTU of 1,460 bytes. You can set a VPC network's MTU to any value between 1,300 bytes and 8,896 bytes (inclusive). Common custom MTU sizes are 1,500 bytes (standard Ethernet) or 8,896 bytes (the maximum possible). We recommend that you configure the MTU for each Compute Engine instance network interface (NIC) to match the MTU of the VPC network to which it is connected. For more information, see Compute instances and MTU settings.

Communication between compute instances within VPC networks

IP packets up to the MTU size can be sent between two compute instances, if the following apply:

• Both the sending and receiving instances use the same VPC network, or peered VPC networks that have identical MTUs.
• The interfaces for both instances are configured to use the VPC network MTU.

To avoid MTU mismatch issues, we recommend that you use the same MTU for all of your connected VPC networks. Although that is the recommended practice, you are not forced to configure identical MTUs on connected VPC networks. For details about how protocols handle situations where there is a MTU mismatch between VPC networks, seeMismatched MTUs, MSS clamping, path MTU discovery.

From the perspective of a sending instance, paths to the following destinations represent instance-to-instance traffic routed within a VPC network:

• A regional internal IPv4 address in a subnet primary IPv4 or subnet secondary IPv4 address range, including private IPv4 address ranges and privately used public IPv4 address ranges, used by these destination resources:
  • The primary internal IPv4 address of a receiving instance's network interface (NIC).
  • An internal IPv4 address in an alias IP range of a receiving instance's NIC.
  • An internal IPv4 address of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
• Internal IPv6 subnet address ranges used by these destination resources:
  • An IPv6 address from the /96 IPv6 address range assigned to a receiving instance's NIC.
  • An IPv6 address from the /96 IPv6 address range of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
• External IPv6 subnet address ranges used by these destination resources when packets are routed using subnet routes or peering subnet routes within the VPC network:
  • An IPv6 address from the /96 IPv6 address range assigned to a receiving instance's NIC.
  • An IPv6 address from the /96 IPv6 address range of an external forwarding rule for either protocol forwarding or for an external passthrough Network Load Balancer.

The following instance-to-instance paths are treated in the same way asCommunication to destinations outside of a VPC network:

• If the packet destination is an external IPv4 address of a receiving instance's NIC.
• If the packet destination is an external IPv4 address of anexternal passthrough Network Load Balancer.
• If the packet destination is an external IPv4 address of a forwarding rule for protocol forwarding
• If the packet destination is an external IPv6 address of an instance's NIC, external passthrough Network Load Balancer, or forwarding rule for external protocol forwarding _and_the applicable route in the VPC network uses a default internet gateway next hop. In this scenario, receiving instances are neither in the same VPC network as the sending instance nor in a VPC network connected to the sending instance's VPC network using VPC Network Peering.

Communication to destinations outside of a VPC network

Google Cloud processes packets sent from compute instances to destinations outside of the sending instance's VPC network as shown in the following table. Destinations outside of a sending instance's VPC network include publicly routable IP addresses for resources outside of Google Cloud and customer-usable external IP addresses within Google Cloud.

Because the internet generally uses an MTU of 1,500 bytes, keeping IP packet size at 1,500 bytes or less usually avoids MTU-related packet loss.

Communication to Google APIs and services

Compute instances using any valid VPC network MTU size can send packets to Google APIs and services, including usingPrivate Google Access andPrivate Service Connect for Google APIs. The details in this section also apply to on-premises resources that send packets to Google APIs and services using Private Google Access for on-premises hosts.

The traffic path to Google APIs and services described in this section is implemented by Google Front Ends (GFEs). These GFEs use non-configurable, fixed MTUs. Traffic from Google Cloud to Google APIs and services always uses the TCP protocol: If a compute instance connects to Google APIs and services from a VPC network whose MTU doesn't match the MTU of the GFE, the segment size is negotiated by using TCP MSS advertisement as described inMismatched MTUs, MSS clamping, path MTU discovery.

Communication through Cloud VPN tunnels

Cloud VPN has both a gateway MTU for encapsulated packets and a_payload MTU_ for packets before and after encapsulation.

For precise payload MTU values and other Cloud VPN MTU information, seeMTU considerationsin the Cloud VPN documentation.

Communication through Cloud Interconnect (VLAN) attachments

We recommend that you use the same MTU for all VLAN attachments that are connected to the same VPC network, and that you set the MTU of the VPC network to the same value. For details about Cloud Interconnect VLAN attachment MTUs, seeCloud Interconnect MTU.

Communication through firewall endpoints

If you use firewall endpoints, configure an appropriate MTU for your VPC network. If the MTU setting of your VPC network exceeds the packet size that your firewall endpoint supports, Cloud Next Generation Firewall can't perform Layer 7 inspection successfully. For more information, see Supported packet size.

Jumbo frame support

Jumbo frames have a payload more than 1,460 bytes in size. The following table summarizes jumbo frame support for Google Cloud products and features:

Compute instances and MTU settings

As a best practice, match a compute instance NIC's MTU to the MTU of the VPC network that the NIC is connected to. The NIC MTU configuration varies depending on the operating system and configuration:

• Linux instances based on a public OS image: each NIC MTU is automatically set to the respective VPC network MTU by using DHCP Option 26.
• Windows instances based on a public OS image: by default, each NIC MTU is configured with a fixed MTU of 1,460 bytes. If you change the MTU of a VPC network that contains Windows instances based on public OS images, you must change the MTU setting of the Windows instances.
• Custom guest OS images: you must configure NIC MTUs or verify that the guest OS accepts the VPC network MTU by using DHCP Option 26.
• Instances with multiple network interfaces: set each NIC MTU to the respective VPC network MTU.

If a NIC MTU must differ from the VPC network MTU, set the NIC MTU to a value that's less than the VPC network MTU. Forcibly decreasing the NIC MTU is advantageous for some advanced networking scenarios.

Changing the MTU of a VPC network

To avoid connectivity issues, before you change the VPC network MTU, you must first stop each compute instance. Rebooting an instance from within its guest operating system doesn't update its MTU. If you choose to configure instances with a lower MTU than the network MTU, the requirement to stop the instance before changing the MTU still applies.

For more information about changing the network MTU, see Change the MTU setting of a VPC network.

GKE and MTU settings

The MTU selected for a Pod interface is dependent on the Container Network Interface (CNI) used by the cluster Nodes and the underlying VPC MTU setting. For more information, seePods.

The Pod interface MTU value is either 1460 or inherited from the primary interface of the Node.

Mismatched MTUs, MSS clamping, path MTU discovery

This section describes how TCP and non-TCP protocols handle mismatched MTUs.

TCP protocol

The TCP protocol handles MTU mismatches automatically. Both the client and server individually calculate their own effective TCP maximum segment size (MSS) values each time a TCP connection is opened. The client and server don't have to agree on an identical effective MSS value.

• Effective TCP maximum segment size (MSS) of the client: The largest amount of transmissible data in a TCP segment sent from a client to a server is the_minimum_ of the following two values:
  • The value of the MSS field in the SYN-ACK packet received by the client from the server during TCP connection establishment.
  • The MTU of the client's network interface, minus 40 bytes. The subtracted 40 bytes includes 20 bytes for the IP header and 20 bytes for the base TCP header.
• Effective TCP maximum segment size (MSS) of the server: The largest amount of transmissible data in a TCP segment sent from a server to a client is the minimum of the following two values:
  • The value of the MSS field in the SYN packet received by the server from the client during TCP connection establishment.
  • The MTU of the server's network interface, minus 40 bytes. The subtracted 40 bytes includes 20 bytes for the IP header and 20 bytes for the base TCP header.

TCP MSS clamping

TCP MSS clamping is a process where a network device between a client and server changes the MSS values in SYN and SYN-ACK packets as it routes the packets between the client and server. Google Cloud uses MSS clamping whenever it sends packets to destinations outside of a VPC network.

Cloud VPN tunnels and Cloud Interconnect VLAN attachments also use MSS clamping. For more information, see Packet encapsulation and processingin the Cloud VPN documentation and Cloud Interconnect MTU.

VPC networks don't perform MSS clamping for packets routed by the next hops within a VPC network because the TCP protocol itself is sufficient.

Non-TCP protocols

Other protocols, such as UDP, require special care when two different VPC network MTUs are involved. It is the responsibility of a sending system to emit packets that fit within its network interface MTU, the MTU of the receiving system's network interface, and the MTU of all networks in between. Google Cloud does not perform IP fragmentation for packets routed by the next hops within a VPC network.

When an IP packet is too large to be delivered—for example, when the packet exceeds the MTU of the VPC network where the receiving compute instance's NIC is located— Google Cloud drops the packet. If the packet has the DF bit set, Google Cloud also sends a Fragmentation Needed (ICMP over IPv4) or Packet Too Big (ICMPv6) message back to the sender.

Google Cloud sends a Fragmentation Needed or Packet Too Big message in the following circumstances, even when a DF bit is off:

• If the VPC network MTU is less than 1,600 bytes, and the packet being sent exceeds the VPC network MTU.
• If the VPC network MTU is 1,600 bytes or more, and the packet being sent exceeds 1,600 bytes.

ICMP Fragmentation Needed or Packet Too Big messages are necessary for a compute instance that is sending packets to use Path MTU discovery (PMTUD). To illustrate how PMTUD works, consider the following example with two compute instances in different VPC networks that are connected by using VPC Network Peering:

• The sending instance has a NIC in a VPC network whose MTU is 8,896 bytes.
• The receiving instance has a NIC in a VPC network whose MTU is 1,460 bytes.
• The sending instance emits an 8,000-byte IP packet whose Don't Fragment (DF) bit is set. Because the packet is too large to be delivered to the receiving instance, Google Cloud sends a Fragmentation Required or Packet Too Big message to the sending instance. This message indicates the largest possible IP packet size that the sender can use when it attempts to re-transmit packets for the connection.
• The sending instance's operating system uses this information to lower the IP packet size when sending subsequent packets to the receiving instance.

PMTUD has the following additional requirements because PMTUD-generated Fragmentation Needed or Packet Too Big packets use the ICMP protocol and have sources that match an original packet's destination:

• You must configure ingress allow VPC firewall rules or rules in firewall policies such that ICMP (for IPv4) or ICMPv6 (for IPv6) are allowed from sources that match the original packet destinations. To simplify firewall configuration, consider allowing ICMP and ICMPv6 from all sources.
• Forwarding rules for internal passthrough Network Load Balancer and internal protocol forwarding must use the L3_DEFAULT protocol so that they process both ICMP for PMTUD and the protocol used by the original packet.

What's next

• To see a different MTU working, see Create and verify a jumbo frame MTU network.
• Create aVPC network with a specified MTU.
• Change the MTU setting of a VPC network.

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how VPC performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try VPC free