Cluster Director overview (original) (raw)
You can deploy, manage, and monitor clusters that you want to run artificial intelligence (AI), machine learning (ML), or high performance computing (HPC) workloads on by using Cluster Director. Cluster Director is a Google Cloud product that automates the complex setup and configuration of clusters, helping you configure compute, networking, and storage resources for your clusters to maximize performance and minimize downtimes.
Cluster Director is designed for IT administrators and AI researchers who want to avoid the overhead of managing a cluster, and instead focus on running their workloads.
Cluster Director key features
Cluster Director integrates multiple Google Cloud services into a cohesive system that simplifies cluster management and provides the following features:
- Automated cluster lifecycle management: the Cluster Director platform is built around a unified control plane with its own API and UI in the Google Cloud console. This feature is a central place to perform all cluster lifecycle operations, letting you programmatically deploy, manage, and scale clusters with all their necessary compute, networking, and storage resources.
- Managed cluster orchestration and scaling: Cluster Director uses a managed Slurm environment for fault-tolerant and scalable job scheduling. This managed environment lets you dynamically scale your cluster by adding and editing machine configurations to an existing cluster to adjust resources based on your workload demands.
- Resilient, performance-optimized infrastructure: clusters are built on a foundation of performance-optimized hardware, including compute-optimized and accelerator-optimized Compute Engine instances. The system uses topology-aware scheduling to colocate compute instances and minimize network latency. It also features built-in resiliency, with autohealing capabilities that automatically detect and replace failed compute instances to help minimize downtimes.
- Integrated observability: Cluster Director provides built-in tools for monitoring the health, topology, and performance of your clusters through a dashboard in the Google Cloud console. These tools give you an at-a-glance view of your cluster's component health.
Cluster Director use cases
Cluster Director is designed to meet the needs of the following use cases:
| Use case | Example workloads |
|---|---|
| Large-scale AI workloads | Generative AI and large language model (LLM) training Fraud detection Recommendation models |
| HPC workloads | Complex simulations Drug discovery, protein folding, and genomic analysis Risk analysis and quantitative trading |
Pricing
There is no charge for using Cluster Director itself. Instead, you incur charges for the underlying Google Cloud resources that your clusters use, such as compute, storage, and networking resources. For more information, see the related pricing documentation:
- Pricing for compute resources
- Pricing for storage services:
- Pricing for networking services