Use signed URLs (original) (raw)

This page provides an overview of signed URLs and instructions for using them with Cloud CDN. Signed URLs give time-limited resource access to anyone in possession of the URL, regardless of whether the user has a Google Account.

A signed URL is a URL that provides limited permission and time to make a request. Signed URLs contain authentication information in their query strings, allowing users without credentials to perform specific actions on a resource. When you generate a signed URL, you specify a user or service account that must have sufficient permission to make the request associated with the URL.

After you generate a signed URL, anyone who possesses it can use the signed URL to perform specified actions (such as reading an object) within a specified period of time.

Signed URLs also support an optional URLPrefix parameter, allowing you to provide access to multiple URLs based on a common prefix.

If you want to scope access to a specific URL prefix, considerusing signed cookies.

Before you begin

Before you use signed URLs, do the following:

For an overview, see Signed URLs and signed cookies.

Configure signed request keys

Creating keys for your signed URLs or signed cookies requires several steps, which are described in the following sections.

Security considerations

Cloud CDN does not validate requests in the following circumstances:

Signed requests must always be validated at the origin before serving the response. This is because origins can be used for serving a mixture of signed and unsigned content and because a client might access the origin directly.

For Cloud Storage backends, make sure toremove public access, so that Cloud Storage can reject requests that are missing a valid signature.

The following table summarizes the behavior.

Request has signature Cache hit Behavior
No No Forward to backend origin.
No Yes Serve from cache.
Yes No Validate signature. If valid, forward to backend origin.
Yes Yes Validate signature. If valid, serve from cache.

Create signed request keys

You enable support for Cloud CDN signed URLs and signed cookies by creating one or more keys on a Cloud CDN-enabled backend service, backend bucket, or both.

For each backend service or backend bucket, you can create and delete keys as your security needs dictate. Each backend can have up to three keys configured at a time. We suggest periodically rotating your keys by deleting the oldest, adding a new key, and using the new key when signing URLs or cookies.

You can use the same key name in multiple backend services and backend buckets because each set of keys is independent of the others. Key names can be up to 63 characters. To name your keys, use the characters A-Z, a-z, 0-9, _ (underscore), and - (hyphen).

When you create keys, be sure to keep them secure because anyone who has one of your keys can create signed URLs or signed cookies that Cloud CDN accepts until the key is deleted from Cloud CDN. The keys are stored on the computer where you generate the signed URLs or signed cookies. Cloud CDN also stores the keys to verify request signatures.

To keep the keys secret, the key values aren't included in responses to any API requests. If you lose a key, you must create a new one.

To create a signed request key, follow these steps.

Console

  1. In the Google Cloud console, go to the Cloud CDN page.
    Go to Cloud CDN
  2. Click the name of the origin that you want to add the key to.
  3. On the Origin details page, click the Edit button.
  4. In the Origin basics section, click Next to open theHost and path rules section.
  5. In the Host and path rules section, click Next to open theCache performance section.
  6. In the Restricted content section, selectRestrict access using signed URLs and signed cookies.
  7. Click Add signing key.
    1. Specify a unique name for the new signing key.
    2. In the Key creation method section, select Automatically generate. Alternatively, click Let me enter, and then specify a signing key value.
      For the former option, copy the automatically generated signing key value to a private file, which you can use tocreate signed URLs.
    3. Click Done.
    4. In the Cache entry maximum age section, enter a value, and then select a unit of time.
  8. Click Done.

gcloud

The gcloud command-line tool reads keys from a local file that you specify. The key file must be created by generating strongly random 128 bits, encoding them with base64, and then replacing the character + with- and replacing the character / with _. For more information, seeRFC 4648. It is vital that the key is strongly random. On a UNIX-like system, you can generate a strongly random key and store it in the key file with the following command:

head -c 16 /dev/urandom | base64 | tr +/ -_ > KEY_FILE_NAME

To add the key to a backend service:

gcloud compute backend-services
add-signed-url-key BACKEND_NAME
--key-name KEY_NAME
--key-file KEY_FILE_NAME

To add the key to a backend bucket:

gcloud compute backend-buckets
add-signed-url-key BACKEND_NAME
--key-name KEY_NAME
--key-file KEY_FILE_NAME

Configure Cloud Storage permissions

If you use Cloud Storage and you have restricted who can read the objects, you must give Cloud CDN permission to read the objects by adding the Cloud CDN service account to Cloud Storage ACLs.

You don't need to create the service account. The service account is created automatically the first time that you add a key to a backend bucket in a project.

Before you run the following command, add at least one key to a backend bucket in your project. Otherwise, the command fails with an error because the Cloud CDN cache fill service account is not created until you add one or more keys for the project.

gcloud storage buckets add-iam-policy-binding gs://BUCKET
--member=serviceAccount:service-PROJECT_NUM@cloud-cdn-fill.iam.gserviceaccount.com
--role=roles/storage.objectViewer

Replace PROJECT_NUM with your project number andBUCKET with your storage bucket.

The Cloud CDN service accountservice-PROJECT_NUM@cloud-cdn-fill.iam.gserviceaccount.comdoesn't appear in the list of service accounts in your project. This is because the Cloud CDN service account is owned by Cloud CDN, not your project.

For more information about project numbers, seeLocate the project ID and project numberin the Google Cloud console Help documentation.

Customize the maximum cache time

Cloud CDN caches responses for signed requests regardless of the backend's Cache-Control header. The maximum time that responses can be cached without revalidation is set by the signed-url-cache-max-age flag, which defaults to one hour and can be modified as shown here.

To set the maximum cache time for a backend service or backend bucket, run one of the following commands:

gcloud compute backend-services update BACKEND_NAME --signed-url-cache-max-age MAX_AGE

gcloud compute backend-buckets update BACKEND_NAME --signed-url-cache-max-age MAX_AGE

List signed request key names

To list the keys on a backend service or backend bucket, run one of the following commands:

gcloud compute backend-services describe BACKEND_NAME

gcloud compute backend-buckets describe BACKEND_NAME

Delete signed request keys

When URLs signed by a particular key should no longer be honored, run one of the following commands to delete that key from the backend service or backend bucket:

gcloud compute backend-services
delete-signed-url-key BACKEND_NAME --key-name KEY_NAME

gcloud compute backend-buckets
delete-signed-url-key BACKEND_NAME --key-name KEY_NAME

Sign URLs

The last step is to sign URLs and distribute them. You can sign URLs by using the gcloud compute sign-url command or by using code that you write yourself. If you need many signed URLs, custom code provides better performance.

Create signed URLs

Use these instructions to create signed URLs by using thegcloud compute sign-url command. This step assumes that you have alreadycreated the keys.

Console

You cannot create signed URLs by using the Google Cloud console. You can use the Google Cloud CLI or write custom code by using the following examples.

gcloud

The Google Cloud CLI includes a command for signing URLs. The command implements the algorithm described in the section aboutwriting your own code.

gcloud compute sign-url
"URL"
--key-name KEY_NAME
--key-file KEY_FILE_NAME
--expires-in TIME_UNTIL_EXPIRATION
[--validate]

This command reads and decodes the base64url encodedkey value from KEY_FILE_NAME, and then outputs a signed URL that you can use for GET or HEAD requests for the given URL.

For example:

gcloud compute sign-url
"https://example.com/media/video.mp4"
--key-name my-test-key
--expires-in 30m
--key-file sign-url-key-file

The URL must be a valid URL that has a path component. For example, http://example.com is invalid, buthttps://example.com/ and https://example.com/whatever are both valid URLs.

If the optional --validate flag is given, this command sends a HEADrequest with the resulting URL, and prints the HTTP response code. If the signed URL is correct, the response code is the same as the result code sent by your backend. If the response code isn't the same, recheckKEY_NAME and the contents of the specified file, and make sure that the value ofTIME_UNTIL_EXPIRATION is at least several seconds.

If the --validateflag is not given, the following are not verified:

Programmatically create signed URLs

The following code samples demonstrate how to programmatically create signed URLs.

Go

Ruby

.NET

Java

Python

PHP

Programmatically create signed URLs with a URL prefix

The following code samples demonstrate how to programmatically create signed URLs with a URL prefix.

Generate custom signed URLs

When you write your own code to generate signed URLs, your goal is to create URLs with the following format or algorithm; all URL parameters are case-sensitive and must be in the order shown:

https://example.com/foo?Expires=EXPIRATION&KeyName=KEY_NAME&Signature=SIGNATURE

To generate signed URLs, follow these steps:

  1. Ensure that the URL for signing does not have a Signature query parameter.
  2. Determine when the URL expires and append an Expires query parameter with the required expiration time in UTC time(the number of seconds since 1970-01-01 00:00:00 UTC). To maximize security, set the value to the shortest time period possible for your use case. The longer a signed URL is valid, the bigger the risk that the user that you give it to shares it with others, accidentally or otherwise.
  3. Set the key name. The URL must be signed with a key of the backend service or backend bucket that serves the URL. It is best to use the most recently added key for key rotation. Add the key to the URL by appending &KeyName=KEY_NAME. ReplaceKEY_NAME with the name of the chosen key created in Creating signed request keys.
  4. Sign the URL. Create the signed URL by following these steps. Make sure that the query parameters are in the order shown immediately before step 1, and make sure that nothing in the signed URL changes case.
    a. Hash the entire URL (including http:// or https:// at the beginning and &KeyName... at the end) with HMAC-SHA1 by using the secret key that corresponds to the key name chosen earlier. Use the raw 16-byte secret key, not the base64url encoded key. Decode it if needed.
    b. Use base64url encode to encode the result.
    c. Append &Signature= to the URL, followed by the encoded signature. Don't convert trailing = characters of the signature to their percent-encoded form, %3D.

Use URL prefixes for signed URLs

Instead of signing the full request URL with the Expires and KeyName query parameters, you can instead sign the URLPrefix, Expires, and KeyName query parameters only. This allows a given combination of URLPrefix, Expires,KeyName, and Signature query parameters to be reused verbatim across multiple URLs that match the URLPrefix, avoiding the need to create a new signature for each distinct URL.

In the following example, the highlighted text shows the parameters that you sign. The Signature is appended as the final query parameter, as usual.

https://media.example.com/videos/id/master.m3u8?userID=abc123&starting_profile=1&URLPrefix=aHR0cHM6Ly9tZWRpYS5leGFtcGxlLmNvbS92aWRlb3Mv&Expires=1566268009&KeyName=mySigningKey&Signature=8NBSdQGzvDftrOIa3WHpp646Iis=

Unlike signing a full request URL, when you sign with URLPrefix you aren't signing any query parameters, so query parameters can be freely included in the URL. And, unlike full request URL signatures, those additional query parameters can appear both before and after the query parameters that make up the signature. As a result, the following is also a valid URL with a signed URL prefix:

https://media.example.com/videos/id/master.m3u8?userID=abc123&URLPrefix=aHR0cHM6Ly9tZWRpYS5leGFtcGxlLmNvbS92aWRlb3Mv&Expires=1566268009&KeyName=mySigningKey&Signature=8NBSdQGzvDftrOIa3WHpp646Iis=&starting_profile=1

URLPrefix denotes a URL-safe base64 encoded URL prefix that encompasses all paths that the signature should be valid for.

A URLPrefix encodes a scheme (either http:// or https://), FQDN, and an optional path. Ending the path with a / is optional but recommended. The prefix shouldn't include query parameters or fragments such as ? or #.

For example, https://media.example.com/videos matches requests to both of the following:

The prefix's path is used as a text substring, not strictly a directory path. For example, the prefix https://example.com/data grants access to both of the following:

To avoid this mistake, we recommend ending all prefixes with / unless you intentionally choose to end the prefix with a partial filename such ashttps://media.example.com/videos/123 to grant access to the following:

If the requested URL doesn't match the URLPrefix, Cloud CDN rejects the request and returns an HTTP 403 error to the client.

Validate signed URLs

The process of validating a signed URL is essentially the same as generating a signed URL. For example, suppose that you want to validate the following signed URL:

https://example.com/PATH?Expires=EXPIRATION&KeyName=KEY_NAME&Signature=SIGNATURE

You can use the secret key named by KEY_NAME to independently generate the signature for the following URL:

https://example.com/PATH?Expires=EXPIRATION&KeyName=KEY_NAME

Then you can verify that it matches SIGNATURE.

Suppose that you want to validate a signed URL that has a URLPrefix, as shown here:

https://example.com/PATH?URLPrefix=URL_PREFIX&Expires=EXPIRATION&KeyName=KEY_NAME&Signature=SIGNATURE

First, verify that the base64-decoded value of URL_PREFIXis a prefix of https://example.com/PATH. If so, you can then calculate the signature for the following:

URLPrefix=URL_PREFIX&Expires=EXPIRATION&KeyName=KEY_NAME

You can then verify that it matches SIGNATURE.

For URL-based signing methods, where the signature is part of the query parameters or embedded as a URL path component, the signature and related parameters are removed from the URL before the request is sent to the origin. This prevents the signature from causing routing issues when the origin is handling the request. To validate these requests, you can inspect thex-client-request-url request header, which includes the original (signed) client request URL prior to the removal of the signed components.

Remove public access to the Cloud Storage bucket

For signed URLs to properly protect content, it is important that the origin server not grant public access to that content. When using a Cloud Storage bucket, a common approach is to make objects public temporarily for testing purposes. After enabling signed URLs, it's important to remove the allUsers (and allAuthenticatedUsers, if applicable) READ permissions (in other words, the Storage Object Viewer Identity and Access Management role) on the bucket.

After you disable public access on the bucket, individual users can still access Cloud Storage without signed URLs if they have access permission, such as OWNER permission.

To remove public allUsers READ access on a Cloud Storage bucket, reverse the action described inMaking all objects in a bucket publicly readable.

Distribute and use signed URLs

The URL returned from the Google Cloud CLI or produced by your custom code can be distributed according to your needs. We recommend signing only HTTPS URLs, because HTTPS provides a secure transport that prevents the Signaturecomponent of the signed URL from being intercepted. Similarly, make sure that you distribute the signed URLs over secure transport protocols such as TLS/HTTPS.