SkyPilot API Server Helm Chart Values — SkyPilot documentation (original) (raw)
The SkyPilot API server helm chart provides typical helm values as configuration entries. Configuration values can be passed in two ways when installing the chart:
Fields#
apiService
#
Configuration for the SkyPilot API server deployment.
apiService.image
#
Docker image to use for the API server.
Default: "berkeleyskypilot/skypilot:0.9.1"
apiService: image: berkeleyskypilot/skypilot:0.9.1
To use a nightly build, find the desired nightly version on pypi and update the image
value:
apiService:
Replace 1.0.0.devYYYYMMDD with the desired nightly version
image: berkeleyskypilot/skypilot-nightly:1.0.0.devYYYYMMDD
apiService.preDeployHook
#
Commands to run before deploying the API server (e.g., install admin policy).
Default: see the yaml below.
apiService: preDeployHook: |- # Run commands before deploying the API server, e.g. installing an admin # policy. Remember to set the admin policy in the config section below. echo "Pre-deploy hook"
# Uncomment the following lines to install the admin policy
# echo "Installing admin policy"
# pip install git+https://github.com/michaelvll/admin-policy-examples
apiService.config
#
Content of the SkyPilot config.yaml to set on the API server. Set to null
to use an empty config. Refer to setting the SkyPilot config for more details.
Default: null
apiService: config: |- allowed_clouds: - aws - gcp
apiService.sshNodePools
#
Content of the ~/.sky/ssh_node_pools.yaml
to set on the API server. Set to null
to use an empty ssh node pools. Refer to Deploy SkyPilot on existing machines for more details.
Default: null
apiService: sshNodePools: |- my-cluster: hosts: - 1.2.3.4 - 1.2.3.5
my-box:
hosts:
- hostname_in_ssh_config
apiService.sshKeySecret
#
Optional secret that contains SSH identity files to the API server to use, all the entries in the secret will be mounted to ~/.ssh/
directory in the API server. Refer to Deploy SkyPilot on existing machines for more details.
Default: null
apiService: sshKeySecret: my-ssh-key-secret
The content of the secret should be like:
apiVersion: v1 kind: Secret metadata: name: my-ssh-key-secret data: id_rsa:
apiService.skipResourceCheck
#
Skip resource check for the API server (not recommended for production), refer to tuning API server resources for more details.
Default: false
apiService: skipResourceCheck: false
apiService.resources
#
Resource requests and limits for the API server container. Refer to tuning API server resources for how to tune the resources.
Default: see the yaml below.
apiService: resources: requests: cpu: "4" memory: "8Gi" limits: cpu: "4" memory: "8Gi"
apiService.skypilotDev
#
Enable developer mode for SkyPilot.
Default: false
apiService: skypilotDev: false
storage
#
storage.enabled
#
Enable persistent storage for the API server, setting this to false
is prone to data loss and should only be used for testing.
Default: true
storage.storageClassName
#
Storage class to use for the API server, leave empty to use the default storage class of the hosting Kubernetes cluster.
Default: ""
storage: storageClassName: gp2
storage.size
#
Size of the persistent storage volume for the API server.
Default: 10Gi
storage.selector
#
Selector for matching specific PersistentVolumes. Usually left empty.
Default: {}
storage.volumeName
#
Name of the PersistentVolume to bind to. Usually left empty to let Kubernetes select and bind the volume automatically.
Default: ""
storage.annotations
#
Annotations to add to the PersistentVolumeClaim.
Default: {}
ingress
#
ingress.enabled
#
Enable ingress for the API server. Set to true
to expose the API server via an ingress controller.
Default: true
ingress.authSecret
#
Name of the Kubernetes secret containing basic auth credentials for ingress. If not specified, a new secret will be created using authCredentials
. This is ignored if ingress.oauth2-proxy.enabled
is true
.
One of ingress.authSecret
or ingress.authCredentials
must be set, unless ingress.oauth2-proxy.enabled
is true
.
Default: null
ingress: authSecret: null
ingress.authCredentials
#
Basic auth credentials in the format username:encrypted_password
. Used only if authSecret
is not set. This is ignored if ingress.oauth2-proxy.enabled
is true
.
One of ingress.authSecret
or ingress.authCredentials
must be set, unless ingress.oauth2-proxy.enabled
is true
.
Default: "username:$apr1$encrypted_password"
ingress: authCredentials: "username:$apr1$encrypted_password"
ingress.path
#
The base path of the API server. You may use different paths to expose multiple API servers through a unified ingress controller.
Default: '/'
ingress.oauth2-proxy
#
Configuration for the OAuth2 Proxy authentication for the API server. This enables SSO providers like Okta.
If enabled, ingress.authSecret
and ingress.authCredentials
are ignored.
Default: see the yaml below.
ingress: oauth2-proxy: enabled: false # Required when enabled: oidc-issuer-url: null client-id: "" client-secret: "" # Optional settings: image: "quay.io/oauth2-proxy/oauth2-proxy:v7.9.0" use-https: false email-domain: "*" session-store-type: "redis" redis-url: null cookie-refresh: null cookie-expire: null
ingress.oauth2-proxy.enabled
#
Enable OAuth2 Proxy for authentication. When enabled, this will deploy an OAuth2 Proxy component and configure the ingress to use it for authentication instead of basic auth.
Default: false
ingress: oauth2-proxy: enabled: true
ingress.oauth2-proxy.oidc-issuer-url
#
The URL of the OIDC issuer (e.g., your Okta domain). Required when oauth2-proxy is enabled.
Default: null
ingress: oauth2-proxy: oidc-issuer-url: "https://mycompany.okta.com"
ingress.oauth2-proxy.client-id
#
The OAuth client ID from your OIDC provider (e.g., Okta). Required when oauth2-proxy is enabled.
Default: ""
ingress: oauth2-proxy: client-id: "0abc123def456"
ingress.oauth2-proxy.client-secret
#
The OAuth client secret from your OIDC provider (e.g., Okta). Required when oauth2-proxy is enabled.
Default: ""
ingress: oauth2-proxy: client-secret: "abcdef123456"
ingress.oauth2-proxy.image
#
Docker image for the OAuth2 Proxy component.
Default: "quay.io/oauth2-proxy/oauth2-proxy:v7.9.0"
ingress: oauth2-proxy: image: "quay.io/oauth2-proxy/oauth2-proxy:v7.9.0"
ingress.oauth2-proxy.use-https
#
Set to true
when using HTTPS for the API server endpoint. When set to false
, secure cookies are disabled, which is required for HTTP endpoints.
Default: false
ingress: oauth2-proxy: use-https: true
ingress.oauth2-proxy.email-domain
#
Email domains to allow for authentication. Use "*"
to allow all email domains.
Default: "*"
ingress: oauth2-proxy: email-domain: "mycompany.com"
ingress.oauth2-proxy.session-store-type
#
Session storage type for OAuth2 Proxy. Can be set to "cookie"
or "redis"
. Using Redis as a session store results in smaller cookies and better performance for large-scale deployments.
Default: "redis"
ingress: oauth2-proxy: session-store-type: "redis"
ingress.oauth2-proxy.redis-url
#
URL to connect to an external Redis instance for session storage. If set to null
and session-store-type
is "redis"
, a Redis instance will be automatically deployed. Format: redis://host[:port][/db-number]
Default: null
ingress: oauth2-proxy: redis-url: "redis://redis-host:6379/0"
ingress.oauth2-proxy.cookie-refresh
#
Duration in seconds after which to refresh the access token. This should typically be set to the access token lifespan minus 1 minute. If not set, tokens will not be refreshed automatically.
Default: null
ingress: oauth2-proxy: cookie-refresh: 3540 # 59 minutes (for a 60-minute access token)
ingress.oauth2-proxy.cookie-expire
#
Expiration time for cookies in seconds. Should match the refresh token lifespan from your OIDC provider.
Default: null
ingress: oauth2-proxy: cookie-expire: 86400 # 24 hours
ingress-nginx
#
ingress-nginx.enabled
#
Enable the ingress-nginx controller for the API server. If you have an existing ingress-nginx controller, you have to set this to false
to avoid conflict.
Default: true
ingress-nginx: enabled: true
ingress-nginx.controller
#
Fields under ingress-nginx.controller
will be mapped to controller
values for the ingress-nginx controller sub-chart. Refer to the ingress-nginx chart documentation for more details.
Default: see the yaml below.
ingress-nginx: controller: service: # Service type of the ingress controller. type: LoadBalancer # Annotations for the ingress controller service. annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" cloud.google.com/l4-rbs: "enabled" service.beta.kubernetes.io/port_443_health-probe_protocol: "TCP" service.beta.kubernetes.io/port_80_health-probe_protocol: "TCP" config: # Custom HTTP snippet to inject into the ingress-nginx configuration. http-snippet: | map httpupgradehttp_upgrade httpupgradeconnection_upgrade { default upgrade; '' close; }
rbac
#
rbac.create
#
Whether to create the service account and RBAC policies for the API server. If false, an external service account is expected.
Default: true
rbac.serviceAccountName
#
Name of the service account to use. Leave empty to let the chart generate one.
Default: ""
rbac: serviceAccountName: ""
rbac.namespaceRules
#
Namespace-scoped RBAC rules granted to the namespace where the SkyPilot tasks will be launched.
Default: see the yaml below.
rbac: namespaceRules: - apiGroups: [ "" ] resources: [ "pods", "pods/status", "pods/exec", "pods/portforward" ] verbs: [ "" ] - apiGroups: [ "" ] resources: [ "services" ] verbs: [ "" ] - apiGroups: [ "" ] resources: [ "secrets" ] verbs: [ "*" ] - apiGroups: [ "" ] resources: [ "events" ] verbs: [ "get", "list", "watch" ]
rbac.clusterRules
#
Cluster-scoped RBAC rules for the API server.
Default: see the yaml below.
rbac: clusterRules: - apiGroups: [ "" ] resources: [ "nodes" ] verbs: [ "get", "list", "watch" ] - apiGroups: [ "" ] resources: [ "pods" ] verbs: [ "get", "list", "watch" ] - apiGroups: [ "node.k8s.io" ] resources: [ "runtimeclasses" ] verbs: [ "get", "list", "watch" ] - apiGroups: [ "networking.k8s.io" ] resources: [ "ingressclasses" ] verbs: [ "get", "list", "watch" ] - apiGroups: ["" ] resources: ["services"] verbs: ["list", "get"]
rbac.manageRbacPolicies
#
Allow the API server to grant permissions to SkyPilot Pods and system components. Refer to setting minimum permissions in helm deployment for more details.
Default: true
rbac: manageRbacPolicies: true
rbac.manageSystemComponents
#
Allow the API server to manage system components in the skypilot-system namespace. Required for object store mounting.
Default: true
rbac: manageSystemComponents: true
kubernetesCredentials
#
kubernetesCredentials.useApiServerCluster
#
Enable using the API server’s cluster for workloads.
Default: true
kubernetesCredentials: useApiServerCluster: true
kubernetesCredentials.useKubeconfig
#
Use the kube-credentials secret containing the kubeconfig to authenticate to Kubernetes.
Default: false
kubernetesCredentials: useKubeconfig: false
kubernetesCredentials.kubeconfigSecretName
#
Name of the secret containing the kubeconfig file. Only used if useKubeconfig is true.
Default: kube-credentials
kubernetesCredentials: kubeconfigSecretName: kube-credentials
kubernetesCredentials.inclusterNamespace
#
Namespace to use for in-cluster resources.
Default: null
kubernetesCredentials: inclusterNamespace: null
awsCredentials
#
awsCredentials.enabled
#
Enable AWS credentials for the API server.
Default: false
awsCredentials: enabled: false
awsCredentials.awsSecretName
#
Name of the secret containing the AWS credentials. Only used if enabled is true.
Default: aws-credentials
awsCredentials: awsSecretName: aws-credentials
awsCredentials.accessKeyIdKeyName
#
Key name used to set AWS_ACCESS_KEY_ID.
Default: aws_access_key_id
awsCredentials: accessKeyIdKeyName: aws_access_key_id
awsCredentials.secretAccessKeyKeyName
#
Key name used to set AWS_SECRET_ACCESS_KEY.
Default: aws_secret_access_key
awsCredentials: secretAccessKeyKeyName: aws_secret_access_key
gcpCredentials
#
gcpCredentials.enabled
#
Enable GCP credentials for the API server.
Default: false
gcpCredentials: enabled: false
gcpCredentials.projectId
#
GCP project ID. Only used if enabled is true.
Default: null
gcpCredentials: projectId: null
gcpCredentials.gcpSecretName
#
Name of the secret containing the GCP credentials. Only used if enabled is true.
Default: gcp-credentials
gcpCredentials: gcpSecretName: gcp-credentials
podSecurityContext
#
Security context for the API server pod. Usually left empty to use defaults. Refer to set the security context for Pod for more details.
Default: {}
podSecurityContext: runAsUser: 1000 runAsGroup: 3000 fsGroup: 2000
securityContext
#
securityContext.capabilities
#
Linux capabilities to drop for the API server container.
Default: drop all capabilities.
securityContext: capabilities: drop: - ALL
securityContext.allowPrivilegeEscalation
#
Whether to allow privilege escalation in the API server container.
Default: false
securityContext: allowPrivilegeEscalation: false
runtimeClassName
#
The runtime class to use for the API server pod. Usually left empty to use the default runtime class.
Default: (empty)