Create and manage cluster and node pool labels (original) (raw)

This page provides an overview of cluster labels and node pool labels in Google Kubernetes Engine (GKE).

GKE cluster and node pool labels and Kubernetes labels

GKE cluster and node pool labels are distinct fromlabels in Kubernetes. The two labeling systems work independently and don't inherit or share labels.

GKE cluster and node pool labels are arbitrary metadata that are attached to your resources. You can use these labels to track information about billing and usage.

Separately, in Kubernetes, the system uses labels internally to associate cluster components and resources (for example, Pods and nodes) with one another and manage resource lifecycles. You can edit Kubernetes labels with the Kubernetes API. You can also use the GKE API to edit Kubernetes labels on your nodes with cluster creationor with a cluster update.

What are cluster labels?

A cluster label is a key-value pair that you can assign to Google Cloud clusters. They help you organize these resources and manage your costs at scale, with the granularity you need. You can attach a label to each resource, then filter the resources based on their labels. Information about labels is forwarded to the billing system that lets you break down your billed charges by label. With built-in billing reports, you can filter and group costs by resource labels. You can also use labels to query billing data exports.

Requirements for cluster labels

The cluster labels applied to a resource must meet the following requirements:

These limits apply to the key and value for each cluster label, and to the individual Google Cloud resources that have cluster labels. There is no limit on how many cluster labels you can apply across all resources within a project.

Common uses of cluster labels

Here are some common use cases for cluster labels:

We don't recommend creating large numbers of unique labels, such as for timestamps or individual values for every API call. The problem with this approach is that when the values change frequently or with keys that clutter the catalog, this makes it difficult to effectively filter and report on resources.

Labels can be used as queryable annotations for resources, but can't be used to set conditions on policies. Tags provide a way to conditionally allow or deny policies based on whether a resource has a specific tag, by providing fine-grained control over policies. For more information, see theTags overview.

Automatically applied labels

GKE automatically applies several labels to cluster resources.

For example, GKE applies labels to Compute Engine instances, persistent disks, and accelerators (TPU).

The following table lists the labels that GKE automatically applies to resources:

Label Applied resources
goog-gke-node Compute Engine VM instance underlying a GKE node.
goog-gke-volume Compute Engine persistent disk attached to a VM instance underlying a GKE node.
goog-gke-tpu Cloud TPU on GKE.
goog-k8s-cluster-name Compute Engine VM instance and persistent disks attached to a VM instance underlying a GKE node.
goog-k8s-cluster-location Compute Engine VM instance and persistent disks attached to a VM instance underlying a GKE node.
goog-k8s-node-pool-name Compute Engine VM instances and their boot disks underlying a GKE node.
goog-fleet-project Compute Engine VM instance and persistent disks attached to a VM instance underlying a GKE node, if the cluster is registered to a fleet.
goog-gke-accelerator-type GKE node pool.
goog-gke-tpu-node-pool-type GKE node pool.
goog-gke-node-pool-provisioning-model GKE node pool.

Do not edit or delete reserved labels. Any changes that you make to reserved labels are reconciled automatically.

Label propagation

In GKE, you can apply labels to clusters and to Standard mode node pools. When you label a cluster, the label propagates to all of the cluster's individual resources such as nodes, instances, and persistent disks. When you label a node pool, the label propagates to all the node pool's individual resources such as instances, and persistent disks. In Standard clusters, if the cluster labels conflict with node pool labels, such as when the label key is the same but the value is different, node pool labels override cluster labels.

Any labels you apply to your clusters or node pools propagate through a background process that runs hourly. It can take up to one hour for a label to appear on all resources associated with a given cluster. In addition, labels only propagate to Compute Engine instances and Persistent Disks that are attached to Compute Engine instances. Other resources such as workloads, forwarding rules, IP addresses, and non-attached Persistent Disks are not labeled.

For Standard clusters, when cluster and node pool labels are applied to Compute Engine resources, you canmanually change the resource labelswithout those changes being reconciled by GKE. In Autopilot clusters, you can't manually modify the labels on the Compute Engine resources because GKE manages the nodes for you. However, we recommend managing these labels through GKE. Existing labels on those Compute Engine resources will not be removed by GKE unless they conflict with the automatically applied labels.

Before you begin

Before you start, make sure you have performed the following tasks:

Get a label fingerprint for API requests

You can skip this step if you use the Google Cloud console or the gcloud CLI.

When you update or add cluster labels using the GKE API, you must provide the latest cluster label fingerprint with your request to prevent any conflicts with other requests.

To get the latest cluster label fingerprint, run a GET request for the appropriate cluster. For example:

GET https://container.googleapis.com/v1/projects/myproject/zones/us-central1-f/clusters/example-cluster

The output is similar to the following:

200 OK

{

 "name": "mycluster",
 "description": "production-cluster",
 "initialNodeCount": 3,
 ...
 "resourceLabels": {
    "environment": "production",
    ...
  },
  "labelFingerprint": "p1ty_9HoBk0="
}

In this output, the labelFingerprint property is the cluster label fingerprint.

You can create Autopilot and Standard clusters with labels by using the gcloud CLI, the Google Cloud console, the GKE API, or Terraform.

gcloud

Run the following command to create an Autopilot cluster with labels:

gcloud container clusters create-auto CLUSTER_NAME \
    --labels=KEY=VALUE

Or, run the following command to create a Standard cluster with labels:

gcloud container clusters create CLUSTER_NAME \
    --labels=KEY=VALUE

Replace the following:

Console

To add labels when creating your cluster, do the following:

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.
    Go to Google Kubernetes Engine
  2. Click Create.
  3. Configure your cluster.
  4. From the navigation pane, under Cluster, click Metadata.
  5. Click Add label.
  6. Add labels.
  7. Click Create.

API

To include a label when creating your cluster, specify the resourceLabelsobject within the cluster object that you provide toprojects.zones.clusters.create.

Terraform

To create an Autopilot cluster with labels using Terraform, refer to the following example:

To create a Standard cluster with labels using Terraform, refer to the following example:

To learn more about using Terraform, seeTerraform support for GKE.

Add or update labels for existing clusters

You can add or update labels for existing Autopilot and Standard clusters using the gcloud CLI, the Google Cloud console, or the GKE API.

gcloud

Run the following command:

gcloud container clusters update CLUSTER_NAME \
    --location =CONTROL_PLANE_LOCATION \
    --update-labels=KEY=VALUE

Replace the following:

The label update overwrites any existing labels on the cluster. If the cluster has existing labels that you want to keep, you must include those labels along with any new labels that you want to add.

Console

To add or update labels, do the following:

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.
    Go to Google Kubernetes Engine
  2. On the Cluster details page, select the checkboxes for one or more clusters that you want to modify.
  3. Click Labels.
  4. Add or update labels.
  5. Click Save.

API

Make a POST request to the cluster'sresourceLabelsmethod with the latest fingerprint and a full list of labels to apply.

Similar to metadata and tags, if the cluster has existing labels you want to keep, you must include those labels in the request along with any new labels that you want to add.

For example, the following snippet makes a request to the resourceLabelsmethod:

POST https://container.googleapis.com/v1/projects/myproject/zones/us-central1-f/clusters/example-cluster/resourceLabels

{
 "resourceLabels": {
  "environment": "production",
  "an-existing-tag": ""
 },
 "labelFingerprint": "42WmSpB8rSM="
}

Remove cluster labels

You can remove labels from Autopilot and Standard clusters using the gcloud CLI, the Google Cloud console, or the GKE API.

gcloud

Run the update command with the --remove-labels flag:

gcloud container clusters update CLUSTER_NAME \
    --remove-labels=KEYS

Replace the following:

Console

To remove labels, do the following:

  1. Go to the Google Kubernetes Engine page in Google Cloud console.
    Go to Google Kubernetes Engine
  2. In the cluster list, select the checkboxes for one or more clusters that you want to modify.
  3. Click Labels.
  4. Click Delete item next to the Value field for the labels that you want to delete.
  5. Click Save.

API

Make a POST request to the resourceLabels method for the appropriate cluster. Provide the current labelsFingerprint and an empty list of labels to remove all labels, or provide a list of labels you want to keep, omitting the labels you want to remove. For example:

Request

POST https://container.googleapis.com/v1/projects/myproject/zones/us-central1-f/clusters/example-cluster/resourceLabels

{
 "resourceLabels": { },
 "labelFingerprint": "42WmSpB8rSM="
}

Create a Standard node pool with labels

You can create a node pool with labels in a Standard cluster by using the gcloud CLI, the Google Cloud console, or the GKE API.

gcloud

Run the following command:

gcloud container node-pools create NODE_POOL_NAME \
    --cluster=CLUSTER_NAME \
    --location=CONTROL_PLANE_LOCATION \
    --labels=KEY=VALUE

Replace the following:

API

To include a label when creating your node pool, specify theresourceLabels object within the node pool object that you provide toprojects.zones.clusters.nodePools.create.

Console

To add labels to new node pool, do the following:

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.
    Go to Google Kubernetes Engine
  2. On the Cluster details page, click the name of the cluster you want to modify.
  3. Click Add Node Pool.
  4. Configure your new node pool.
  5. From the navigation pane, click Metadata.
  6. Click Add label.
  7. Add labels.
  8. Click Create.

Add or update labels for an existing node pool

You can add or update labels for existing node pools using the gcloud CLI, the Google Cloud console, or the GKE API.

This change requires recreating the nodes, which can cause disruption to your running workloads. For details about this specific change, find the corresponding row in the manual changes that recreate the nodes using a node upgrade strategy without respecting maintenance policiestable. To learn more about node updates, see Planning for node update disruptions.

gcloud

Run the following command:

gcloud container node-pools update NODE_POOL_NAME \
    --cluster=CLUSTER_NAME \
    --location=CONTROL_PLANE_LOCATION \
    --labels=KEY=VALUE

Replace the following:

The label update overwrites any existing labels on the node pool. If the node pool has existing labels that you want to keep, you must include those labels along with any new labels that you want to add.

API

To include a label when modifying your node pool, specify the resourceLabelsobject within the node pool object that you provide toprojects.zones.clusters.nodePools.update.

Console

To add or update labels, do the following:

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.
    Go to Google Kubernetes Engine
  2. In the cluster list, click the name of the cluster that contains the node pool that you want to modify.
  3. In the Node Pools section, click the name of the node pool that you want to modify.
  4. Click Edit
  5. In the Labels section, add or update labels.
  6. Click Save.

Remove node pool labels

You can remove labels from node pools using the gcloud CLI, the Google Cloud console, or the GKE API.

gcloud

Run the following command:

gcloud container node-pools update NODE_POOL_NAME \
    --cluster=CLUSTER_NAME \
    --location=CONTROL_PLANE_LOCATION \
    --labels=KEY=VALUE

Replace the following:

The label update overwrites any existing labels on the node pool. If the node pool has existing labels that you want to keep, you must exclude any labels that you want to remove. If you want to remove all labels, use--labels= without any key-value pairs.

API

To include a label when modifying your node pool, specify theresourceLabels object within the node pool object that you provide toprojects.zones.clusters.nodePools.update. Provide an empty list of labels to remove all labels, or provide a list of labels you want to keep. Omit the labels you want to remove.

Console

To remove labels from a node pool, do the following

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.
    Go to Google Kubernetes Engine
  2. In the cluster list, click the name of the cluster that contains the node pool that you want to modify.
  3. In the Node Pools section, click the name of the node pool that you want to modify.
  4. Click Delete item next to theValue field for the labels that you want to delete.
  5. Click Save.

What's next

Read the GKE overview.