Use cross-bucket replication (original) (raw)

This page describes how to use cross-bucket replication, which usesStorage Transfer Service to copy new and updated objects asynchronously from a source bucket to a destination bucket. When you use cross-bucket replication, you create and manage replication jobs, which are a type of job in Storage Transfer Service.

Before you begin

To use cross-bucket replication, you must complete prerequisite steps. For instructions, refer to Before you begin in the Storage Transfer Service documentation.

Create a replication job

Console

When you create a cross-bucket replication job using the Google Cloud console, the Storage Transfer Service service agent is automatically granted the storage.legacyBucketOwner role on the source bucket and the storage.legacyBucketWriter role on the destination bucket.

When using the Google Cloud console, you can create a replication job for existing buckets or for new buckets during the bucket creation process.

To create a replication job for a new bucket, follow the instructions forcreating a new bucket.

To create a replication job for an existing bucket, complete the following steps:

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.
    Go to Buckets
  2. In the list of buckets, click the name of the source bucket whose objects you want to replicate.
  3. On the Bucket details page, click the Configuration tab.
  4. Locate the Cross-bucket replication option and clickEdit.
  5. In the Edit cross-bucket replication pane that opens, clickAdd a destination.
  6. In the Choose a destination section, select a destination bucket, then click Next.
  7. In the Choose replication settings section, do the following:
    • Optional: To filter objects to replicate by object name prefix, select the Replicate objects based on prefix checkbox in theChoose which objects to replicate section.
      * To include objects by prefix, enter a prefix in theInclude objects with prefix section, then click Add a prefix.
      * To exclude objects by prefix, enter a prefix in theExclude objects with prefix section, then clickAdd a prefix.
    • Optional: To set a storage class for replicated objects, select a storage class from the menu in theSet storage class for replicated objects section.
      If you skip this step, replicated objects use the destination bucket's storage class by default.
    1. Click Save.

Command line

For Google Cloud CLI instructions on how to create a replication job, refer to Create a replication job using the Google Cloud CLI.

REST APIs

To check the status of the replication job,view Cloud Logging for Storage Transfer Service logs.

List replication jobs

Console

You cannot list replication jobs using the Google Cloud console. SeeView replication job for instructions on how to view a single replication job at a time.

Command line

For Google Cloud CLI instructions on how to list replication jobs, refer to List replication jobs using the Google Cloud CLI.

REST APIs

View a replication job

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.
    Go to Buckets
  2. In the list of buckets, click the name of the source bucket whose cross-bucket replication job you want to view.
  3. On the Bucket details page, click the Configuration tab.
  4. Locate the Cross-bucket replication option and clickEdit.
    The Edit cross-bucket replication pane appears, which displays the replication job for each destination bucket.

On the Buckets page, you can view the Replication column, which displays whether a bucket has a Turbo replication job or a cross-bucket replication job running. For instructions on displaying the Replicationcolumn, see Show columns.

Command line

For Google Cloud CLI instructions on how to view a replication job, refer to View a replication job using the Google Cloud CLI.

REST APIs

Update a replication job

You can update the following fields of a replication job:

Console

When using the Google Cloud console, you can only update a replication job by pausing or unpausing the job.

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.
    Go to Buckets
  2. In the list of buckets, click the name of the source bucket that you want to pause or unpause replicating.
  3. On the Bucket details page, click the Configuration tab.
  4. Locate the Cross-bucket replication option and clickEdit.
  5. In the Edit cross-bucket replication pane that appears, clickPause orUnpause next to the replication job you want to update.

Command line

For Google Cloud CLI instructions on how to update a replication job, refer to Update a replication job using the Google Cloud CLI.

REST APIs

Delete a replication job

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.
    Go to Buckets
  2. In the list of buckets, click the name of the source bucket you want to stop replicating.
  3. On the Bucket details page, click the Configuration tab.
  4. Locate the Cross-bucket replication option and clickEdit.
  5. In the Edit cross-bucket replication pane that appears, clickDelete next to the replication job you want to delete.
  6. In the dialogue that appears, click Confirm.

Command line

For Google Cloud CLI instructions on how to delete a replication job, refer to Delete a replication job using the Google Cloud CLI.

REST APIs

What's next

Monitor your replicated objects.