Access Google APIs through endpoints (original) (raw)

This document explains how to use Private Service Connect endpoints to connect to Google APIs. Instead of sending API requests to the publicly available IP addresses for service endpoints such asstorage.googleapis.com, you can send the requests to the internal IP address of an endpoint.

You can also use Private Service Connect to access services in another VPC networkand to publish services.

Required roles

To get the permissions that you need to manage endpoints that access global Google APIs, ask your administrator to grant you the following IAM roles:

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to manage endpoints that access global Google APIs. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to manage endpoints that access global Google APIs:

You might also be able to get these permissions with custom roles or other predefined roles.

Before you begin

Enable Private Google Access for a subnet

VMs without an external IP address assigned must be connected to a subnet with Private Google Access enabled to access Google APIs and services using an endpoint.

If the VM has more than one interface, connect the interface that is configured with a default route (usually nic0).

The source IP address of packets sent from the VM must match the VM interface's primary internal IPv4 address or an internal IPv4 address from an alias IP range.

To enable Private Google Access on a subnet, follow these steps.

Console

  1. In the Google Cloud console, go to the VPC networks page.
    Go to VPC networks
  2. Click the name of the network that contains the subnet for which you need to enable Private Google Access.
  3. Click the name of the subnet. The Subnet details page is displayed.
  4. Click Edit.
  5. In the Private Google Access section, select On.
  6. Click Save.

gcloud

  1. Determine the name and region of the subnet. To list the subnets for a particular network, use the following command:
    gcloud compute networks subnets list --filter=NETWORK_NAME
  2. Run the following command to enable Private Google Access:
    gcloud compute networks subnets update SUBNET_NAME \
    --region=REGION \
    --enable-private-ip-google-access
  3. Verify that Private Google Access is enabled by running this command:
    gcloud compute networks subnets describe SUBNET_NAME \
    --region=REGION \
    --format="get(privateIpGoogleAccess)"

Replace the following:

Terraform

You can use the Terraform resourceto enable Private Google Access on a subnet.

To learn how to apply or remove a Terraform configuration, seeBasic Terraform commands.

Create an endpoint

After you have chosen an IP address thatmeets the requirements, you can create an endpoint.

An endpoint connects to Google APIs and services using a global forwarding rule. Each forwarding rule counts toward theper VPC network quota for Private Service Connect.

You can't update an endpoint for Google APIs and services after it is created. If you need to update an endpoint for Google APIs and services, delete the endpoint, and then create a new one.

Permissions required for this task

To perform this task, you must have been granted the following permissions_or_ one of the following IAM roles.

Permissions for the gcloud CLI and the API

Permissions for the Google Cloud console

Roles

See Roles for role information.

Console

  1. In the Google Cloud console, go to the Private Service Connect page.
    Go to Private Service Connect
  2. Click the Connected endpoints tab.
  3. Click Connect endpoint.
  4. For Target, select the target API bundlethat you want to use:
    • All Google APIs
    • VPC-SC
  5. For Endpoint name, enter a name for the endpoint.
  6. Select a Network for the endpoint.
  7. Select an IP Address for the endpoint.
    The IP address must meet these requirements.
    If you need a new IP address, you can create one:
    1. Click Create IP address.
    2. Enter a Name and Description for the IP address.
    3. Enter the IP address you want to use and click Save.
  8. If a Service Directory region is not already configured for this VPC network, select the region you want to use.
    All endpoints that are used to access Google APIs and services in a given VPC network use the same Service Directory region.
  9. If a Service Directory namespace is not already configured for this VPC network, configure the namespace you want to use:
    • To use an automatically assigned namespace, click the Namespacemenu and select the automatically assigned namespace.
    • To select an existing namespace that is used in another network, click the Namespace drop-down menu and select a namespace from the list. The list displays all namespaces in the project. You must select a namespace that is used only for endpoints that are used to access Google APIs.
    • To create a new namespace, click the Namespace drop-down menu and click Create namespace. Enter the namespace and clickCreate.
      All endpoints that you use to access Google APIs and services in a given VPC network use the same Service Directory namespace.
  10. Click Add endpoint.

gcloud

  1. Reserve a global internal IP address to assign to the endpoint.
    gcloud compute addresses create ADDRESS_NAME \
    --global \
    --purpose=PRIVATE_SERVICE_CONNECT \
    --addresses=ENDPOINT_IP \
    --network=NETWORK_NAME
    Replace the following:
    • ADDRESS_NAME: the name to assign to the reserved IP address.
    • ENDPOINT_IP: the IP address to reserve for the endpoint.
      The IP address must meet these requirements.
    • NETWORK_NAME: the name of the VPC network for the endpoint.
  2. Create a forwarding rule to connect the endpoint to Google APIs and services. The --service-directory-registration flag is optional. If you omit it, an existing namespace is reused, or if there are no existing namespaces, a system-generated namespace is assigned.
    gcloud compute forwarding-rules create ENDPOINT_NAME \
    --global \
    --network=NETWORK_NAME \
    --address=ADDRESS_NAME \
    --target-google-apis-bundle=API_BUNDLE \
    --service-directory-registration=REGION_NAMESPACE_URI
    Replace the following:
    • ENDPOINT_NAME: the name to assign to the endpoint. The name must be a string of 1-20 characters, containing only lower-case letters and numbers. The name must start with a letter.
    • NETWORK_NAME: the name of the VPC network for the endpoint.
    • ADDRESS_NAME: the name of the reserved address on the associated network.
    • API_BUNDLE: the bundle of APIs to make available using the endpoint. See the list of supported APIs.
      * Use all-apis to give access to all supported APIs.
      * Use vpc-sc to restrict access to Google APIs that support VPC Service Controls.
    • REGION_NAMESPACE_URI: the URI of the Service Directory region or namespacethat you want to use. This URI must reference the same project that you are creating the endpoint in.
      * You can define a region only withprojects/PROJECT_NAME/locations/REGION.
      * You can define a region and namespace withprojects/PROJECT_NAME/locations/REGION/namespaces/NAMESPACE.
      If you omit --service-directory-registration completely, or set a region without a namespace, the following occurs:
      * If a region or namespace is already configured for this VPC network, those defaults are used.
      * If a region is not configured, the region is set tous-central1. If a namespace is not configured, a system-generated namespace is assigned.

API

  1. Reserve a global internal IP address to assign to the endpoint.
    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/addresses
    {
    "name": ADDRESS_NAME,
    "address": ENDPOINT_IP,
    "addressType": "INTERNAL",
    "purpose": PRIVATE_SERVICE_CONNECT,
    "network": NETWORK_URL
    }
    Replace the following:
    • PROJECT_ID: your project ID.
    • ADDRESS_NAME: the name to assign to the reserved IP address.
    • ENDPOINT_IP: the IP address to reserve for the endpoint.
      The IP address must meet these requirements.
    • NETWORK_URL: the VPC network for the endpoint.
  2. Create a forwarding rule to connect the endpoint to Google APIs and services. The serviceDirectoryRegistrations object is optional. If you omit it, an existing namespace is reused, or if there are no existing namespaces, a system-generated namespace is assigned.
    POST https://compute.googleapis.com/compute/v1/projects/PROJECT/global/forwardingRules
    {
    "name": "ENDPOINT_NAME",
    "IPAddress": "projects/PROJECT/global/addresses/ADDRESS_NAME",
    "network": "projects/PROJECT/global/networks/NETWORK",
    "target": "API_BUNDLE",
    "serviceDirectoryRegistrations": [
    {
    "service_directory_region": REGION,
    "namespace": "NAMESPACE"
    }
    ],
    }
    Replace the following:
    • PROJECT: your project ID.
    • ENDPOINT_NAME: the name to assign to the endpoint. The name must be a string of 1-20 characters, containing only lower-case letters and numbers. The name must start with a letter.
    • NETWORK: the VPC network for the endpoint.
    • ADDRESS_NAME: the name of the reserved address on the associated network.
    • API_BUNDLE: the bundle of APIs to make available using the endpoint. See the list of supported APIs.
      * Use all-apis to give access to all supported APIs.
      * Use vpc-sc to restrict access to Google APIs that support VPC Service Controls.
    • REGION: the Service Directory regionyou want to use. For example, us-central1. If you omitREGION, and a region is already configured for this VPC network, that region is used. If a region is not configured, the region is set to us-central1.
    • NAMESPACE: the name of the Service Directory namespacethat you want to use. If you omit NAMESPACE, and a namespace is already configured for this VPC network, that namespace is used. If a namespace is not configured, a system-generated namespace is assigned.

Terraform

You can use the following Terraform resources to create an endpoint:

Verify that the endpoint is working

Create a VM instance in the VPC network where Private Service Connect is configured. Run the following command on the VM to verify that the Private Service Connect endpoint is working. Endpoints don't respond to ping (ICMP) requests.

curl -v ENDPOINT_IP/generate_204

Replace ENDPOINT_IP with the IP address of the endpoint.

If the endpoint is working, you see an HTTP 204 response code.

List endpoints

You can list all configured endpoints.

Permissions required for this task

To perform this task, you must have been granted the following permissions_or_ one of the following IAM roles.

Permissions for the gcloud CLI and the API

Permissions for the Google Cloud console

Roles

See Roles for role information.

Console

  1. In the Google Cloud console, go to the Private Service Connect page.
    Go to Private Service Connect
  2. Click the Connected endpoints tab.
    The endpoints are displayed.

gcloud

gcloud compute forwarding-rules list
--filter target="(all-apis OR vpc-sc)" --global

The output is similar to the following:

NAME REGION IP_ADDRESS IP_PROTOCOL TARGET RULE IP TCP all-apis

Get information about an endpoint

You can view all the configuration details of an endpoint.

Permissions required for this task

To perform this task, you must have been granted the following permissions_or_ one of the following IAM roles.

Permissions for the gcloud CLI and the API

Permissions for the Google Cloud console

Roles

See Roles for role information.

Console

  1. In the Google Cloud console, go to the Private Service Connect page.
    Go to Private Service Connect
  2. Click the Connected endpoints tab.
    The endpoints are displayed.
  3. Click the endpoint that you want to view details for.

gcloud

gcloud compute forwarding-rules describe
ENDPOINT_NAME --global

Label an endpoint

You can manage labels for endpoints. Seelabeling resources for more information.

Permissions required for this task

To perform this task, you must have been granted the following permissions_or_ one of the following IAM roles.

Permissions

Roles

See Roles for role information.

Delete an endpoint

You can delete an endpoint.

Permissions required for this task

To perform this task, you must have been granted the following permissions_or_ one of the following IAM roles.

Permissions for the gcloud CLI and the API

Permissions for the Google Cloud console

Roles

See Roles for role information.

Console

  1. In the Google Cloud console, go to the Private Service Connect page.
    Go to Private Service Connect
  2. Click the Connected endpoints tab.
  3. Select the endpoint that you want to delete, and click Delete.

gcloud

gcloud compute forwarding-rules delete \
    ENDPOINT_NAME --global

Replace ENDPOINT_NAME with the name of the endpoint that you want to delete.

Use an endpoint

To use an endpoint, you send requests to a DNS hostname that resolves to the IP address of the endpoint.

Use p.googleapis.com DNS names

When you create an endpoint, Service Directory creates DNS records for commonly used APIs and services that are available using the endpoint. DNS records are created only for APIs and services that have default DNS names that end with googleapis.com, and only for a subset of those APIs and services.

The DNS records are created in a p.googleapis.com private zone. The records point to the endpoint IP address, and use this format:SERVICE-ENDPOINT.p.googleapis.com

For example, if your endpoint name is xyz, DNS records are created forstorage-xyz.p.googleapis.com, compute-xyz.p.googleapis.com, and other supported APIs.

Clients that can be configured to use a custom endpoint can use thep.googleapis.com DNS names to send requests to an endpoint.

See the documentation for your client or client library for information about configuring it to use custom endpoints. For example:

Create DNS records by using default DNS names

You need to create DNS records to direct the default DNS names for APIs and services to your endpoint in these circumstances:

To create DNS records that point to your Private Service Connect endpoint, follow these instructions:

  1. Create a DNS zone for the domain you need to use (for example,googleapis.com or gcr.io). Consider creating a Cloud DNS private zone for this purpose.
  2. In this DNS zone:
    1. Create an A record for the domain (zone) name itself; for example,googleapis.com or gcr.io. Point this A record to the IP address of the endpoint. If you're using Cloud DNS, see adding a record.
    2. Create a CNAME record for all of the additional domain's possible host names by using an asterisk and a dot followed by the domain (zone) name; for example, *.googleapis.com or *.gcr.io. Point this CNAME record to theA record in the same zone. For example, point *.googleapis.com togoogleapis.com or point *.gcr.io to gcr.io.

Access the endpoint from on-premises hosts

If your on-premises network is connected to a VPC network, you can use Private Service Connect to access Google APIs and services from on-premises hosts by using the internal IP address of the endpoint.

Troubleshooting

The following sections contain information about resolving issues with Private Service Connect endpoints that are used to access Google APIs.

Private DNS zone creation fails

When you create an endpoint, a Service Directory DNS zone is created. Zone creation can fail for these reasons:

Conflicting zones might exist because a previous deletion failed.

To create the Service Directory DNS zone, do the following:

  1. Verify that the Cloud DNS API isenabled in your project.
  2. Verify that you have the required permissions to create the Service Directory DNS zone:
    • dns.managedZones.create
    • servicedirectory.namespaces.associatePrivateZone
  3. Delete the DNS zone.
  4. Create a Service Directory DNS zonebacked by the Service Directory namespace associated with your endpoint.
    Use the following values when you create the zone:
    • Zone name: Use the same zone name that the system used during the failed creation attempt. The error message displays what zone name was used.
    • DNS name: p.googleapis.com. (include the trailing dot).
    • Service Directory namespace: Find the Service Directory namespace for the Private Service Connect endpoint you created, and use this namespace when you create the Service Directory DNS zone.
      The Service Directory namespace has the following format:goog-psc-NETWORK_NAME-NETWORK_ID.

Private DNS zone deletion fails

When you delete the last endpoint in a VPC network, the associated Service Directory configuration including the DNS zone is deleted.

This deletion can fail for these reasons:

To resolve this issue, do the following:

  1. Verify that you have the dns.managedZones.delete permission. For more information, see Access Control in the Cloud DNS documentation.
  2. Delete the DNS zone.