Using VPC Service Controls (VPC SC) (original) (raw)

VPC Service Controls is a Google Cloud feature that lets you set up a secure perimeter to guard against data exfiltration. Both the default run.app URL and custom domains are subject to VPC Service Controls. This guide shows how to use VPC Service Controls with Cloud Run to add additional security to your services.

Limitations and advisories

For Artifact Registry or Container Registry:
- The registry where you store your container must be in the same VPC Service Controls service perimeter as the project you're deploying to.
- The code being built must be in the same perimeter as the registry that the container is being pushed to.
The continuous deploymentfeature is not available for projects inside a VPC Service Controls perimeter.
When Cloud Run services are invoked, VPC Service Controls policy enforcement does not use the client's IAM authentication information. Such requests have the following limitations:
- VPC Service Controls ingress policy rules that use IAM principals are not supported.
- Access levels for VPC Service Controls perimeters that use IAM principals are not supported.

Set up your organization to support VPC Service Controls

To use VPC Service Controls with Cloud Run, you can configure a service perimeter at the organization level. By configuring appropriate organization policies, you can make sure that VPC Service Controls checks are enforced when using Cloud Run and that developers can only deploy services compliant with VPC Service Controls. Learn more aboutinheritanceand violationswhen setting an organization policy.

If you are deploying Cloud Run functions with gcloud functions, you or your administrator must grant the Cloud Build service account access on the VPC Service Controls perimeter. Refer to Build process overviewfor more information.

Set up a VPC Service Controls perimeter

The following steps show how toset up a VPC Service Controls perimeter.

Make sure that you have therequired roles for administering VPC Service Controls.
In the Google Cloud console, go to the VPC Service Controls page.
Go to the VPC Service Controls page
If you are prompted, select your Organization.
Click Manage Access Policies. Select an access policy orcreate one. The access policy must include all projects that you want to add to the perimeter.
Click New Perimeter. Type a name for the perimeter.
Select the projects that you want to secure within the perimeter.
1. Click Projects.
2. In the Add Projects window, select the projects you want to add.
  If you are using Shared VPC, make sure to add the host project and service projects.
3. Click the Add Projects button.
Select Cloud Run as the service that you want to secure within the perimeter.
1. Click Restricted Services.
2. Click Add Services.
3. Search for "Cloud Run." Then selectCloud Run Admin API. (Select Cloud Functions V2 API if you used gcloud functions.)
4. Click Add Cloud Run Admin API (orCloud Functions V2 API if you used gcloud functions).
Enable access for developers.
Turning on VPC Service Controls for Cloud Run prevents all access from outside the perimeter, including viewing and deploying services from machines not in the perimeter, such as company laptops. In order for developers and application operators to view and deploy resources within the perimeter, you must configure the perimeteringress policy:
1. Click Ingress Policy.
2. Specify the Source, Identity, Project, and Services.
3. Click Add Rule.
  If your organization uses Access Context Manager, you can also enable access for developers by setting access levels.
Click Create Perimeter.

Once you have completed these steps, all calls to the Cloud Run Admin API are checked to make sure that they originate from within the same perimeter.

Set up organization policies

To manage organization policies, you need theOrganization Policy Administrator(roles/orgpolicy.policyAdmin) role.

To comply with VPC Service Controls and protect against data exfiltration, set up the following organization policies that control the allowableingress and egresssettings for Cloud Run in the service perimeter.

Restrict allowed ingress settings

The run.allowedIngress organization policy controls theingress settingsthat developers are allowed to use for Cloud Run. Set this organization policy to mandate that developers use the valueinternal if you want to allow only internal requests, orinternal-and-cloud-load-balancing if you want to allow requests through an external Application Load Balancer, with the access controls managed by Cloud Load Balancing.

For more information on these settings, refer toRestricting ingress for Cloud Run

Console

Go to the Allowed ingress settings (Cloud Run) policy page in the Google Cloud console:
Go to organization policy
Click Edit.
On the Edit page, select Customize.
Under Policy enforcement, select Replace.
Under Policy values, select Custom.
Under Policy type, select Allow.
Under Custom values, enter internal. If you'd like to also allow Cloud Load Balancing traffic, click New policy value and enterinternal-and-cloud-load-balancing.
Click Save.

gcloud

Use the gcloud resource-manager org-policies allow command:

gcloud resource-manager org-policies allow
run.allowedIngress internal
--organization ORGANIZATION_ID

where ORGANIZATION_ID is your organization ID. If you'd like to also allow Cloud Load Balancing traffic, replacerun.allowedIngress internal with run.allowedIngress internal-and-cloud-load-balancing.

After this organization policy is in place, all services must use allowed values for their ingress settings. This means that Cloud Run services can only accept traffic originating from a VPC network within the service perimeter. Attempts to set ingress to a different value will be blocked by the organization policy.

Restrict allowed VPC egress settings

The run.allowedVPCEgress organization policy controls theegress settingsthat developers are allowed to use for Cloud Run. Set this organization policy to allow only the value all-traffic:

Console

Go to the Allowed VPC egress settings (Cloud Run) policy page in the Google Cloud console:
Go to organization policy
Click Edit.
On the Edit page, select Customize.
Under Policy enforcement, select Replace.
Under Policy values, select Custom.
Under Policy type, select Allow.
Under Custom values, enter all-traffic.
Click Save.

gcloud

Use the gcloud resource-manager org-policies allow command:

gcloud resource-manager org-policies allow
run.allowedVPCEgress all-traffic
--organization ORGANIZATION_ID

where ORGANIZATION_ID is your organization ID.

Behavior after organization policy is set

After the organization policy is in place, all new revisions must use Direct VPC egress or a Serverless VPC Access connector and must use the value all-traffic for their egress settings. This combination routes all egress traffic from these revisions through a VPC network, subjecting this traffic to the VPC network's firewall rules and other settings. New revision deployments specifying a different value are blocked by the organization policy.

Org policy and traffic migrations

For services existing before the organization policy was set, you can continue to migrate traffic to non-compliant revisions until all revisions serving traffic have become compliant.

This behavior lets you test and gradually roll out new revisions that comply with the policy. After all revisions serving traffic are compliant with the policy, any attempt to move traffic to non-compliant revisions will be blocked by the org policy.

Set up your project to support VPC Service Controls

For individual projects within the service perimeter, you must perform additional configuration to use VPC Service Controls.

Configure VPC networks

This section shows how to configure your VPC network so that requests sent to the regular googleapis.com domains are automatically routed to the restricted restricted virtual IP (VIP) range,199.36.153.4/30 (restricted.googleapis.com) where your Cloud Run service is serving. No code changes are needed.

For each VPC network in a project, follow these steps to block outbound traffic except for traffic to the restricted VIP range:

Enable Private Google Accesson the subnets hosting your VPC network resources.
Configure firewall rulesto prevent data from leaving the VPC network:
- Create a deny egress rule that blocks all outbound traffic.
- Create an allow egress rule that permits traffic to 199.36.153.4/30on TCP port 443. Make sure that it has a priority before the deny egress rule you just created—this allows egress only to the restricted VIP range.
Create a Cloud DNS response policy.
gcloud dns response-policies create RESPONSE_POLICY \
--networks=NETWORK \
--description=DESCRIPTION
Replace the following:
- RESPONSE_POLICY: a name for your response policy.
- NETWORK: the name of the network to associate with this policy. To specify multiple networks, separate the names of each network by commas, for example: network1,network2.
- DESCRIPTION: a description of the response policy, such as Cloud Run VPC Service Controls response policy
Add a rule to the response policy to resolve *.googleapis.com torestricted.googleapis.com. The IP address range forrestricted.googleapis.com is 199.36.153.4/30.
gcloud dns response-policies rules create RESPONSE_POLICY_RULE \
--response-policy=RESPONSE_POLICY \
--dns-name="*.googleapis.com." \
--local-data=name="restricted.googleapis.com.",type="A",ttl=300,rrdatas="199.36.153.4|199.36.153.5|199.36.153.6|199.36.153.7"
Replace the following:
- RESPONSE_POLICY_RULE: a name for your response policy rule.
- RESPONSE_POLICY: the name of the response policy that you created in the previous step, in quotes.
Add a rule to the response policy to resolve *.run.app (or *.cloudfunctions.netif you created your function using gcloud functions deploy) to therestricted.googleapis.com. The IP address range forrestricted.googleapis.com is 199.36.153.4/30.
gcloud dns response-policies rules create RESPONSE_POLICY_RULE \
--response-policy=RESPONSE_POLICY \
--dns-name="*.run.app." \
--local-data=name="restricted.googleapis.com.",type="A",ttl=300,rrdatas="199.36.153.4|199.36.153.5|199.36.153.6|199.36.153.7"
Replace the following:
- RESPONSE_POLICY_RULE: a name for your response policy rule.
- RESPONSE_POLICY: the name of the response policy that you specified in the previous step, in quotes.

At this point, requests originating from within the VPC network:

Are not able to leave the VPC network, preventing egress outside the service perimeter.
Can only reach Google APIs and services that check VPC Service Controls, preventing exfiltration through Google APIs.

Deploy VPC Service Controls-compliant Cloud Run services and jobs

After VPC Service Controls is configured for Cloud Run, make sure that all Cloud Run services and jobs deployed within the service perimeter enable VPC Service Controls. This requires the following:

**Cloud Run services and jobs must use Direct VPC egress or a Serverless VPC Access connector.**See Connecting to a VPC networkfor more information.
**Cloud Run services and jobs must route all outgoing traffic through the VPC network.**See Egress settingsfor more information.
Cloud Run services must allow traffic from internal sources only. SeeIngress settingsfor more information. Setting ingress to "all" disables VPC Service Controls enforcement.

Audit existing services for VPC Service Controls compliance

After setting up VPC Service Controls, new services created in projects within the service perimeter are automatically checked for compliance. However, to avoid disruption of existing workloads, existing services continue to work and might not comply with organization policies.

Google recommends that you periodically audit your services to make sure your ingress and egress settings conform with the requirements, and update or redeploy services as necessary. For example, you could create a script that uses the Cloud Run Admin API to list your services and highlight those that do not specify the proper network settings.