SkyPilot API Server Helm Chart Values — SkyPilot documentation (original) (raw)

The SkyPilot API server helm chart provides typical helm values as configuration entries. Configuration values can be passed in two ways when installing the chart:

Fields#

apiService#

Configuration for the SkyPilot API server deployment.

apiService.image#

Docker image to use for the API server.

Default: "berkeleyskypilot/skypilot:0.9.1"

apiService: image: berkeleyskypilot/skypilot:0.9.1

To use a nightly build, find the desired nightly version on pypi and update the image value:

apiService:

Replace 1.0.0.devYYYYMMDD with the desired nightly version

image: berkeleyskypilot/skypilot-nightly:1.0.0.devYYYYMMDD

apiService.preDeployHook#

Commands to run before deploying the API server (e.g., install admin policy).

Default: see the yaml below.

apiService: preDeployHook: |- # Run commands before deploying the API server, e.g. installing an admin # policy. Remember to set the admin policy in the config section below. echo "Pre-deploy hook"

# Uncomment the following lines to install the admin policy
# echo "Installing admin policy"
# pip install git+https://github.com/michaelvll/admin-policy-examples

apiService.config#

Content of the SkyPilot config.yaml to set on the API server. Set to null to use an empty config. Refer to setting the SkyPilot config for more details.

Default: null

apiService: config: |- allowed_clouds: - aws - gcp

apiService.sshNodePools#

Content of the ~/.sky/ssh_node_pools.yaml to set on the API server. Set to null to use an empty ssh node pools. Refer to Deploy SkyPilot on existing machines for more details.

Default: null

apiService: sshNodePools: |- my-cluster: hosts: - 1.2.3.4 - 1.2.3.5

my-box:
  hosts:
    - hostname_in_ssh_config

apiService.sshKeySecret#

Optional secret that contains SSH identity files to the API server to use, all the entries in the secret will be mounted to ~/.ssh/ directory in the API server. Refer to Deploy SkyPilot on existing machines for more details.

Default: null

apiService: sshKeySecret: my-ssh-key-secret

The content of the secret should be like:

apiVersion: v1 kind: Secret metadata: name: my-ssh-key-secret data: id_rsa:

apiService.skipResourceCheck#

Skip resource check for the API server (not recommended for production), refer to tuning API server resources for more details.

Default: false

apiService: skipResourceCheck: false

apiService.resources#

Resource requests and limits for the API server container. Refer to tuning API server resources for how to tune the resources.

Default: see the yaml below.

apiService: resources: requests: cpu: "4" memory: "8Gi" limits: cpu: "4" memory: "8Gi"

apiService.skypilotDev#

Enable developer mode for SkyPilot.

Default: false

apiService: skypilotDev: false

storage#

storage.enabled#

Enable persistent storage for the API server, setting this to false is prone to data loss and should only be used for testing.

Default: true

storage.storageClassName#

Storage class to use for the API server, leave empty to use the default storage class of the hosting Kubernetes cluster.

Default: ""

storage: storageClassName: gp2

storage.size#

Size of the persistent storage volume for the API server.

Default: 10Gi

storage.selector#

Selector for matching specific PersistentVolumes. Usually left empty.

Default: {}

storage.volumeName#

Name of the PersistentVolume to bind to. Usually left empty to let Kubernetes select and bind the volume automatically.

Default: ""

storage.annotations#

Annotations to add to the PersistentVolumeClaim.

Default: {}

ingress#

ingress.enabled#

Enable ingress for the API server. Set to true to expose the API server via an ingress controller.

Default: true

ingress.authSecret#

Name of the Kubernetes secret containing basic auth credentials for ingress. If not specified, a new secret will be created using authCredentials. This is ignored if ingress.oauth2-proxy.enabled is true.

One of ingress.authSecret or ingress.authCredentials must be set, unless ingress.oauth2-proxy.enabled is true.

Default: null

ingress: authSecret: null

ingress.authCredentials#

Basic auth credentials in the format username:encrypted_password. Used only if authSecret is not set. This is ignored if ingress.oauth2-proxy.enabled is true.

One of ingress.authSecret or ingress.authCredentials must be set, unless ingress.oauth2-proxy.enabled is true.

Default: "username:$apr1$encrypted_password"

ingress: authCredentials: "username:$apr1$encrypted_password"

ingress.path#

The base path of the API server. You may use different paths to expose multiple API servers through a unified ingress controller.

Default: '/'

ingress.oauth2-proxy#

Configuration for the OAuth2 Proxy authentication for the API server. This enables SSO providers like Okta.

If enabled, ingress.authSecret and ingress.authCredentials are ignored.

Default: see the yaml below.

ingress: oauth2-proxy: enabled: false # Required when enabled: oidc-issuer-url: null client-id: "" client-secret: "" # Optional settings: image: "quay.io/oauth2-proxy/oauth2-proxy:v7.9.0" use-https: false email-domain: "*" session-store-type: "redis" redis-url: null cookie-refresh: null cookie-expire: null

ingress.oauth2-proxy.enabled#

Enable OAuth2 Proxy for authentication. When enabled, this will deploy an OAuth2 Proxy component and configure the ingress to use it for authentication instead of basic auth.

Default: false

ingress: oauth2-proxy: enabled: true

ingress.oauth2-proxy.oidc-issuer-url#

The URL of the OIDC issuer (e.g., your Okta domain). Required when oauth2-proxy is enabled.

Default: null

ingress: oauth2-proxy: oidc-issuer-url: "https://mycompany.okta.com"

ingress.oauth2-proxy.client-id#

The OAuth client ID from your OIDC provider (e.g., Okta). Required when oauth2-proxy is enabled.

Default: ""

ingress: oauth2-proxy: client-id: "0abc123def456"

ingress.oauth2-proxy.client-secret#

The OAuth client secret from your OIDC provider (e.g., Okta). Required when oauth2-proxy is enabled.

Default: ""

ingress: oauth2-proxy: client-secret: "abcdef123456"

ingress.oauth2-proxy.image#

Docker image for the OAuth2 Proxy component.

Default: "quay.io/oauth2-proxy/oauth2-proxy:v7.9.0"

ingress: oauth2-proxy: image: "quay.io/oauth2-proxy/oauth2-proxy:v7.9.0"

ingress.oauth2-proxy.use-https#

Set to true when using HTTPS for the API server endpoint. When set to false, secure cookies are disabled, which is required for HTTP endpoints.

Default: false

ingress: oauth2-proxy: use-https: true

ingress.oauth2-proxy.email-domain#

Email domains to allow for authentication. Use "*" to allow all email domains.

Default: "*"

ingress: oauth2-proxy: email-domain: "mycompany.com"

ingress.oauth2-proxy.session-store-type#

Session storage type for OAuth2 Proxy. Can be set to "cookie" or "redis". Using Redis as a session store results in smaller cookies and better performance for large-scale deployments.

Default: "redis"

ingress: oauth2-proxy: session-store-type: "redis"

ingress.oauth2-proxy.redis-url#

URL to connect to an external Redis instance for session storage. If set to null and session-store-type is "redis", a Redis instance will be automatically deployed. Format: redis://host[:port][/db-number]

Default: null

ingress: oauth2-proxy: redis-url: "redis://redis-host:6379/0"

ingress.oauth2-proxy.cookie-refresh#

Duration in seconds after which to refresh the access token. This should typically be set to the access token lifespan minus 1 minute. If not set, tokens will not be refreshed automatically.

Default: null

ingress: oauth2-proxy: cookie-refresh: 3540 # 59 minutes (for a 60-minute access token)

ingress.oauth2-proxy.cookie-expire#

Expiration time for cookies in seconds. Should match the refresh token lifespan from your OIDC provider.

Default: null

ingress: oauth2-proxy: cookie-expire: 86400 # 24 hours

ingress-nginx#

ingress-nginx.enabled#

Enable the ingress-nginx controller for the API server. If you have an existing ingress-nginx controller, you have to set this to false to avoid conflict.

Default: true

ingress-nginx: enabled: true

ingress-nginx.controller#

Fields under ingress-nginx.controller will be mapped to controller values for the ingress-nginx controller sub-chart. Refer to the ingress-nginx chart documentation for more details.

Default: see the yaml below.

ingress-nginx: controller: service: # Service type of the ingress controller. type: LoadBalancer # Annotations for the ingress controller service. annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" cloud.google.com/l4-rbs: "enabled" service.beta.kubernetes.io/port_443_health-probe_protocol: "TCP" service.beta.kubernetes.io/port_80_health-probe_protocol: "TCP" config: # Custom HTTP snippet to inject into the ingress-nginx configuration. http-snippet: | map httpupgradehttp_upgrade httpupgradeconnection_upgrade { default upgrade; '' close; }

rbac#

rbac.create#

Whether to create the service account and RBAC policies for the API server. If false, an external service account is expected.

Default: true

rbac.serviceAccountName#

Name of the service account to use. Leave empty to let the chart generate one.

Default: ""

rbac: serviceAccountName: ""

rbac.namespaceRules#

Namespace-scoped RBAC rules granted to the namespace where the SkyPilot tasks will be launched.

Default: see the yaml below.

rbac: namespaceRules: - apiGroups: [ "" ] resources: [ "pods", "pods/status", "pods/exec", "pods/portforward" ] verbs: [ "" ] - apiGroups: [ "" ] resources: [ "services" ] verbs: [ "" ] - apiGroups: [ "" ] resources: [ "secrets" ] verbs: [ "*" ] - apiGroups: [ "" ] resources: [ "events" ] verbs: [ "get", "list", "watch" ]

rbac.clusterRules#

Cluster-scoped RBAC rules for the API server.

Default: see the yaml below.

rbac: clusterRules: - apiGroups: [ "" ] resources: [ "nodes" ] verbs: [ "get", "list", "watch" ] - apiGroups: [ "" ] resources: [ "pods" ] verbs: [ "get", "list", "watch" ] - apiGroups: [ "node.k8s.io" ] resources: [ "runtimeclasses" ] verbs: [ "get", "list", "watch" ] - apiGroups: [ "networking.k8s.io" ] resources: [ "ingressclasses" ] verbs: [ "get", "list", "watch" ] - apiGroups: ["" ] resources: ["services"] verbs: ["list", "get"]

rbac.manageRbacPolicies#

Allow the API server to grant permissions to SkyPilot Pods and system components. Refer to setting minimum permissions in helm deployment for more details.

Default: true

rbac: manageRbacPolicies: true

rbac.manageSystemComponents#

Allow the API server to manage system components in the skypilot-system namespace. Required for object store mounting.

Default: true

rbac: manageSystemComponents: true

kubernetesCredentials#

kubernetesCredentials.useApiServerCluster#

Enable using the API server’s cluster for workloads.

Default: true

kubernetesCredentials: useApiServerCluster: true

kubernetesCredentials.useKubeconfig#

Use the kube-credentials secret containing the kubeconfig to authenticate to Kubernetes.

Default: false

kubernetesCredentials: useKubeconfig: false

kubernetesCredentials.kubeconfigSecretName#

Name of the secret containing the kubeconfig file. Only used if useKubeconfig is true.

Default: kube-credentials

kubernetesCredentials: kubeconfigSecretName: kube-credentials

kubernetesCredentials.inclusterNamespace#

Namespace to use for in-cluster resources.

Default: null

kubernetesCredentials: inclusterNamespace: null

awsCredentials#

awsCredentials.enabled#

Enable AWS credentials for the API server.

Default: false

awsCredentials: enabled: false

awsCredentials.awsSecretName#

Name of the secret containing the AWS credentials. Only used if enabled is true.

Default: aws-credentials

awsCredentials: awsSecretName: aws-credentials

awsCredentials.accessKeyIdKeyName#

Key name used to set AWS_ACCESS_KEY_ID.

Default: aws_access_key_id

awsCredentials: accessKeyIdKeyName: aws_access_key_id

awsCredentials.secretAccessKeyKeyName#

Key name used to set AWS_SECRET_ACCESS_KEY.

Default: aws_secret_access_key

awsCredentials: secretAccessKeyKeyName: aws_secret_access_key

gcpCredentials#

gcpCredentials.enabled#

Enable GCP credentials for the API server.

Default: false

gcpCredentials: enabled: false

gcpCredentials.projectId#

GCP project ID. Only used if enabled is true.

Default: null

gcpCredentials: projectId: null

gcpCredentials.gcpSecretName#

Name of the secret containing the GCP credentials. Only used if enabled is true.

Default: gcp-credentials

gcpCredentials: gcpSecretName: gcp-credentials

podSecurityContext#

Security context for the API server pod. Usually left empty to use defaults. Refer to set the security context for Pod for more details.

Default: {}

podSecurityContext: runAsUser: 1000 runAsGroup: 3000 fsGroup: 2000

securityContext#

securityContext.capabilities#

Linux capabilities to drop for the API server container.

Default: drop all capabilities.

securityContext: capabilities: drop: - ALL

securityContext.allowPrivilegeEscalation#

Whether to allow privilege escalation in the API server container.

Default: false

securityContext: allowPrivilegeEscalation: false

runtimeClassName#

The runtime class to use for the API server pod. Usually left empty to use the default runtime class.

Default: (empty)