Using AMD GPUs with minikube (original) (raw)

This tutorial shows how to start minikube with support for AMD GPUs.

Support is provided by the AMD GPU device plugin for Kubernetes.

Prerequisites

• Linux
• Latest AMD GPU Drivers 6.2.1 or greater
• minikube v1.35.0 or later (docker driver only)

Using the docker driver

• Ensure you have an AMD driver installed, you can check if one is installed by running rocminfo, if one is not installed follow the Radeon™ Driver Installation Guide
• Delete existing minikube (optional)
  If you have an existing minikube instance, you may need to delete it if it was built before installing the AMD drivers.
• Start minikube:

minikube start --driver docker --container-runtime docker --gpus amd  

Verifying the GPU is available

Test the AMD GPUs are available to the cluster.

1. Create the following Job:

cat <<'EOF' | kubectl apply -f -  
apiVersion: batch/v1  
kind: Job  
metadata:  
  name: amd-gpu-check  
  labels:  
    purpose: amd-gpu-check  
spec:  
  ttlSecondsAfterFinished: 100  
  template:  
    spec:  
      restartPolicy: Never  
      securityContext:  
        supplementalGroups:  
        - 44  
        - 110  
      containers:  
        - name: amd-gpu-checker  
          image: rocm/rocm-terminal  
          workingDir: /root  
          command: ["rocminfo"]  
          args: []  
          resources:  
            limits:  
              amd.com/gpu: 1 # requesting a GPU  
EOF  

1. Check the Job output kubectl logs jobs/amd-gpu-check looks something like the following:

ROCk module version 6.8.5 is loaded  
=====================  
HSA System Attributes  
=====================  
Runtime Version:         1.14  
Runtime Ext Version:     1.6  
System Timestamp Freq.:  1000.000000MHz  
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)  
Machine Model:           LARGE  
System Endianness:       LITTLE  
Mwaitx:                  DISABLED  
DMAbuf Support:          YES  
==========  
HSA Agents  
==========  
*******  
Agent 1  
*******  
  Name:                    AMD Ryzen 7 7840U w/ Radeon  780M Graphics  
  Uuid:                    CPU-XX  
...  

Where can I learn more about GPU passthrough?

See the excellent documentation athttps://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

Why does minikube not support AMD GPUs on Windows?

minikube supports Windows host through Hyper-V or VirtualBox.

• VirtualBox doesn’t support PCI passthrough for Windows host.
• Hyper-V supports DDA (discrete device assignment) but only for Windows Server 2016

Since the only possibility of supporting GPUs on minikube on Windows is on a server OS where users don’t usually run minikube, we haven’t invested time in trying to support GPUs on minikube on Windows.