Storage Intelligence (original) (raw)
This document describes Storage Intelligence, a Cloud Storage capability that simplifies data management at scale. Storage Intelligence offers a unified platform for data exploration, cost optimization, security enforcement, and governance implementation. Storage Intelligence provides insights into your data through Data storage insights with Gemini assistance. Storage Insights datasets and Storage Insights inventory reports further enhance your analytical capabilities. Based on your analysis, you can take actions, such as relocating buckets and performing large-scale batch operations.
To learn how to configure Storage Intelligence, see Configure and manage Storage Intelligence.
Benefits
Storage Intelligence offers the following benefits:
- Simplifies storage management tasks by providing a single platform to manage, analyze, and optimize Cloud Storage resources.
- Enhances data exploration by providing a centralized platform to search, browse, and analyze data across Cloud Storage locations.
- Analyzes Cloud Storage usage to identify cost saving areas like unused resources, cheaper storage class options, and provides action recommendations to reduce Cloud Storage costs.
- Provides a centralized platform to manage organizational security policies across Cloud Storage environment.
- Provides prebuilt tools and functionalities that minimize development time and the potential for errors. This eliminates the need for custom management solutions.
Key features
Storage Intelligence offers the following types of features:
- Analysis features to understand your data
- Action features to manage your data
Analysis features
Storage Intelligence provides the following analysis features that help you understand your data:
- Data storage insights with Gemini assistance: Use AI-powered assistance through Gemini to help you better understand your data storage landscape. When you use Storage Insights datasets, you can ask Gemini prompts about your bucket and object metadata and usage, such as "5 largest buckets without Autoclass enabled" and "Objects in Standard storage class smaller than 50 MB". For more information, seeAnalyze your stored data with Gemini assistance.
- Storage Insights datasets: Create datasets that provide insights into your Cloud Storage environment. Storage Insights datasets helps you with data exploration, cost optimization, security enforcement, and governance implementation. To use it, you select the scope of your data by choosing the project, folder, or organization that contains the buckets and objects you want to manage. Storage Insights then creates a queryable metadata index of all included buckets and objects. This index is made available to you as a BigQuery linked dataset. Storage Insights datasets is an exclusive feature only available through the Storage Intelligence subscription. For more information about Storage Insights datasets, see datasets.
- Storage Insights inventory reports: Generate per-bucket inventory reports to gain detailed insights into its objects. The Storage Insights inventory reports feature is included as part of the Storage Intelligence subscription which includes any object scan fees. You can also use inventory reports without a subscription to Storage Intelligence. For more information about Storage Insights inventory reports, see inventory reports.
Action features
Storage Intelligence offers the following action features to manage your data effectively:
- Bucket relocation: Bucket relocation facilitates serverless migration of Cloud Storage buckets across geographic locations with minimal downtime. Using bucket relocation, you can relocate buckets while preserving the bucket and object metadata, including bucket names and storage classes. Bucket relocation minimizes application downtime and reduces operational effort. With bucket relocation, you can architect the Cloud Storage workload to meet performance and cost requirements. Bucket relocation is an exclusive feature only available through the Storage Intelligence subscription. For more information about bucket relocation, see bucket relocation.
- Storage batch operations: Perform operations at scale across billions of Cloud Storage objects. You can automate large-scale object operations, eliminating the need for custom script development and maintenance. Storage batch operations is an exclusive feature only available through the Storage Intelligence subscription. For more information, see storage batch operations.
Configuration options
To use Storage Intelligence, you'll need to configure the following settings:
Resource
You can configure Storage Intelligence at different levels of your Google Cloud resource hierarchy. This provides flexibility in controlling which resources Storage Intelligence manages. Configuring Storage Intelligence for a resource automatically configures it for all buckets within that resource, and consequently, for all objects within those buckets. The following sections describe the resources you can configure Storage Intelligence on. For information about how to configure Storage Intelligence for a resource, see Configure Storage Intelligence.
Organization
When you configure Storage Intelligence on an organization, all current and new folders and projects within that organization inherit the organization's configuration. If you have the appropriate permissions, you can exclude folders or projects from inheriting the organization's configuration.
Folder
Configuring Storage Intelligence for a folder is useful for projects that are grouped within folders. When you configure Storage Intelligence on a folder, all current and new projects within that folder inherit the folder's configuration. If you have the appropriate permissions, you can exclude projects from inheriting the folder's configuration.
Project
Configuring Storage Intelligence on a project is suitable for workloads, departments, or teams that are organized by project.
Bucket filters
You can control which Cloud Storage buckets Storage Intelligence manages. You can include or exclude buckets based on their location or name using regular expressions. This granular control helps you limit Storage Intelligence features to only the required buckets. For information about how to apply bucket filters, seeConfigure Storage Intelligence.
Filter by location
Filtering by location lets you choose the Google Cloud locations to include or exclude from Storage Intelligence. Including specific locations focuses your analysis and can reduce processing time. Excluding locations lets you omit data that's regulated or irrelevant to your analysis. This targeted approach streamlines analysis and ensures Storage Intelligence gives you the most relevant and useful insights.
Filter buckets using regular expressions
Filtering by name in Storage Intelligence enables granular control over the scope of analysis by including or excluding buckets using regular expressions.
Edition configuration
The edition configuration determines which Storage Intelligence features are active for a resource. Each resource (organization, folder, or project) has one of the following edition configurations:
- INHERIT: Inherit the configuration of the parent resource.
- STANDARD: Configure the
STANDARDedition of Storage Intelligence. TheSTANDARDedition includes all Storage Intelligence features and is the default edition when you configure Storage Intelligence for a resource. - DISABLED: Disable Storage Intelligence for the specified resource. Child resources inherit the disabled state unless you explicitly set them to
STANDARDorINHERIT. - TRIAL: Configure the 30-day introductory trial for Storage Intelligence.
The following diagram shows an example of edition configurations across different levels of the resource hierarchy:
Figure 1. Edition configurations across different levels of resource hierarchy.
The following table shows how to use edition configurations to configure Storage Intelligence across your Google Cloud resources. When you configure Storage Intelligence on an organization, the projects and folders within the organization inherit the Storage Intelligence setting. However, recognizing that different teams or departments might have unique requirements, you can also create exceptions and fine-tune these configurations at lower levels for specific folders and projects.
| Resource | Edition configuration | Storage Intelligence status | Description |
|---|---|---|---|
| Organization A | STANDARD | Enabled | STANDARD means that Storage Intelligence is enabled for the whole organization. As a result, by default, all projects and folders within the organization A inherit Storage Intelligence. |
| Folder C | DISABLED | Disabled | To disable Storage Intelligence for folder C, you can update the edition configuration to DISABLED. |
| Project E | DISABLED | Disabled | To disable Storage Intelligence for project E, you can update the edition configuration to DISABLED. |
Pricing
Charges for Storage Intelligence are based on the capabilities you configure. For details, see the Storage Intelligence pricing page.