Dataplex pricing (original) (raw)
Dataplex Universal Catalog pricing
Dataplex Universal Catalog pricing is based on pay-as-you-go usage. Dataplex Universal Catalog currently charges based on the following SKUs:
- Dataplex Universal Catalog processing (standard and premium)
- Metadata storage
The following is a high-level overview of how each key Dataplex Universal Catalog capability is billed:
Gemini-powered features in Dataplex including data insights and automated metadata generation features are billed as part of Gemini in BigQuery or Gemini Code Assist (link: https://cloud.google.com/products/gemini/pricing#gemini-in-bigquery-pricing)
Other usage
Data organization features in Dataplex Universal Catalog (lake, zone, or asset setup) and security policy application and propagation, are provided free of charge.
In addition, some Dataplex Universal Catalog functionalities (including scheduled data quality and data ingestion tasks, and Dataplex Universal Catalog managed connectors for ingesting metadata from CloudSQL and Looker) trigger job execution using Dataproc Serverless, BigQuery, Dataflow, and Cloud Scheduler. Those usages are charged according to the Dataproc, BigQuery, Dataflow, and Cloud Scheduler pricing models respectively, and charges will show up under Dataproc, BigQuery, and Dataflow instead of Dataplex Universal Catalog.
Dataplex Universal Catalog processing pricing
Dataplex Universal Catalog standard and premium processing are metered by the Data Compute Unit (DCU). DCU-hour is an abstract billing unit for Dataplex Universal Catalog and the actual metering depends on the individual features you use.
Dataplex Universal Catalog standard processing pricing
Dataplex Universal Catalog standard tier covers the data discovery functionality that discovers metadata across Dataplex Universal Catalog managed data. The following are the prices as per the region of your choice.
- Johannesburg (africa-south1)
- Taiwan (asia-east1)
- Hong Kong (asia-east2)
- Tokyo (asia-northeast1)
- Osaka (asia-northeast2)
- Seoul (asia-northeast3)
- Mumbai (asia-south1)
- Singapore (asia-southeast1)
- Jakarta (asia-southeast2)
- Bangkok (asia-southeast3)
- Sydney (australia-southeast1)
- Melbourne (australia-southeast2)
- Warsaw (europe-central2)
- Finland (europe-north1)
- Stockholm (europe-north2)
- Madrid (europe-southwest1)
- Belgium (europe-west1)
- Berlin (europe-west10)
- Turin (europe-west12)
- London (europe-west2)
- Frankfurt (europe-west3)
- Netherlands (europe-west4)
- Zurich (europe-west6)
- Milan (europe-west8)
- Paris (europe-west9)
- Doha (me-central1)
- Dammam (me-central2)
- Tel Aviv (me-west1)
- Montreal (northamerica-northeast1)
- Toronto (northamerica-northeast2)
- Mexico (northamerica-south1)
- Sao Paulo (southamerica-east1)
- Santiago (southamerica-west1)
- Iowa (us-central1)
- Oklahoma (us-central2)
- South Carolina (us-east1)
- Northern Virginia (us-east4)
- Columbus (us-east5)
- Dallas (us-south1)
- Oregon (us-west1)
- Los Angeles (us-west2)
- Salt Lake City (us-west3)
- Las Vegas (us-west4)
| Item | Meter | Default* (USD) | BigQuery CUD - 1 Year* (USD) | BigQuery CUD - 3 Year* (USD) |
|---|---|---|---|---|
| Dataplex processing | per DCU per unit time | 0.06∣0.06 | 0.06∣0.054 | $0.048 |
* Each consumption model has a unique ID. You may need to opt-in to be eligible for consumption model discounts. Click here to learn more.
Dataplex Universal Catalog free tier
As part of the Google Cloud Free Tier, Dataplex Universal Catalog offers some resources free of charge up to a specific limit. These free usage limits are available during and after the free trial period. If you go over these usage limits and are no longer in the free trial period, you will be charged according to the pricing as described in the sections above.
Note: The Dataplex Universal Catalog free tier is only available for the Dataplex Universal Catalog Standard Processing SKU, and is not available for the Dataplex Universal Catalog Premium Processing SKU.
| Resource | Monthly free usage limits |
|---|---|
| Dataplex Universal Catalog Processing | 100 DCU-hour |
Dataplex Universal Catalog premium processing pricing
The Dataplex premium processing tier covers data lineage, data quality, and data profiling.
- Johannesburg (africa-south1)
- Taiwan (asia-east1)
- Hong Kong (asia-east2)
- Tokyo (asia-northeast1)
- Osaka (asia-northeast2)
- Seoul (asia-northeast3)
- Mumbai (asia-south1)
- Delhi (asia-south2)
- Singapore (asia-southeast1)
- Jakarta (asia-southeast2)
- Bangkok (asia-southeast3)
- Sydney (australia-southeast1)
- Melbourne (australia-southeast2)
- Warsaw (europe-central2)
- Finland (europe-north1)
- Stockholm (europe-north2)
- Madrid (europe-southwest1)
- Belgium (europe-west1)
- Berlin (europe-west10)
- Turin (europe-west12)
- London (europe-west2)
- Frankfurt (europe-west3)
- Netherlands (europe-west4)
- Zurich (europe-west6)
- Milan (europe-west8)
- Paris (europe-west9)
- Doha (me-central1)
- Dammam (me-central2)
- Tel Aviv (me-west1)
- Montreal (northamerica-northeast1)
- Toronto (northamerica-northeast2)
- Mexico (northamerica-south1)
- Sao Paulo (southamerica-east1)
- Santiago (southamerica-west1)
- Iowa (us-central1)
- Oklahoma (us-central2)
- South Carolina (us-east1)
- Northern Virginia (us-east4)
- Columbus (us-east5)
- Alabama (us-east7)
- Dallas (us-south1)
- Oregon (us-west1)
- Los Angeles (us-west2)
- Salt Lake City (us-west3)
- Las Vegas (us-west4)
- Phoenix (us-west8)
| Item | Meter | Default* (USD) | BigQuery CUD - 1 Year* (USD) | BigQuery CUD - 3 Year* (USD) |
|---|---|---|---|---|
| Dataplex premium processing pricing | per DCU per unit time | 0.089∣0.089 | 0.089∣0.0801 | $0.0712 |
* Each consumption model has a unique ID. You may need to opt-in to be eligible for consumption model discounts. Click here to learn more.
Calculation of DCU charges
DCU charges for each feature are calculated as follows:
1. Auto data quality scans:
- The DCU-hour consumption is proportional to the processing involved in profiling the data and computing the data quality metrics. This is billed per second, with a minimum of one minute.
- The charge depends on the number of rows, the number of columns, the amount of data that you've scanned, the data quality rule configuration, the partitioning and clustering settings on the table, and the frequency of the scan.
2. There are several options to reduce the cost of auto data quality scans:
- Sampling
- Incremental scans
- To separate data quality charges from other charges in the Dataplex premium processing SKU, on the Cloud Billing report, use the label goog-dataplex-workload-type with value DATA_QUALITY.
3. To filter aggregate charges, use the following labels available in billing export in BigQuery:
- goog-dataplex-datascan-data-source-dataplex-entity
- goog-dataplex-datascan-data-source-dataplex-lake
- goog-dataplex-datascan-data-source-dataplex-zone
- goog-dataplex-datascan-data-source-project
- goog-dataplex-datascan-data-source-region
- goog-dataplex-datascan-id
- goog-dataplex-datascan-job-id
4. Data Profiling scans:
- The DCU-hour consumption is proportional to the processing involved in profiling the data and computing the data quality metrics. This is billed per second, with a minimum of one minute.
- The charge depends on the number of rows, numbers of columns, the amount of data scanned, partitioning and clustering settings on the table, and the frequency of the scan.
5. There are several options to reduce the cost of data profiling scans:
- Sampling
- Incremental scans
- Column filtering
- Row filtering
- To separate data profiling charges from other charges in the Dataplex premium processing SKU, on the Cloud Billing report, use the label goog-dataplex-workload-type with value DATA_PROFILE.
6. To filter aggregate charges, use the following labels available in billing export in BigQuery:
- goog-dataplex-datascan-data-source-dataplex-entity
- goog-dataplex-datascan-data-source-dataplex-lake
- goog-dataplex-datascan-data-source-dataplex-zone
- goog-dataplex-datascan-data-source-project
- goog-dataplex-datascan-data-source-region
- goog-dataplex-datascan-id
- goog-dataplex-datascan-job-id
7. Data Lineage:
- The DCU-hour consumption is proportional to the processing involved to automatically parse lineage.
- To separate data lineage charges from other charges in the Dataplex premium processing SKU, on the Cloud Billing report, use the label goog-dataplex-workload-type with value LINEAGE.
- If you call the Data Lineage API Origin sourceType with a value other than CUSTOM, it causes additional costs.
Data lineage pricing example
User A enables data lineage to track lineage for BigQuery in their project. The project is in the us-central1 location. During one month, data lineage consumes 100 DCU-hours of Dataplex Premium processing, and generates 1GiB of data lineage metadata. The cost is:
- Example
What's next
- Read the product documentation: Dataplex Universal Catalog, Data Catalog.
- Get started with Dataplex Universal Catalog.
- Learn about Dataplex Universal Catalog solutions and use cases.
Request a custom quote
With Google Cloud's pay-as-you-go pricing, you only pay for the services you use. Connect with our sales team to get a custom quote for your organization.