High-Performance Kubernetes

High-Performance Kubernetes (HPK), allows HPC users to run their own private "mini Clouds" on a typical HPC cluster. HPK uses a single container to run the Kubernetes control plane and a Virtual Kubelet Provider implementation to translate container lifecycle management commands from Kubernetes-native to Slurm/Apptainer.

To allow users to run HPK, the HPC environment should have Apptainer configured so that:

• It allows users to run containers with --fakeroot.
• It uses a CNI plug-in that hands over private IPs to containers, which are routable across cluster hosts (we use flannel and the flannel CNI plug-in).

In contrast to a typical Kubernetes installation at the Cloud:

• HPK uses a pass-through scheduler, which assigns all pods to the single hpk-kubelet that represents the cluster. In practice, this means that all scheduling is delegated to Slurm.
• All Kubernetes services are converted to headless. This avoids the need for internal, virtual cluster IPs that would need special handling at the network level. As a side effect, HPK services that map to multiple pods are load-balanced at the DNS level if clients support it.

HPK is a continuation of the KNoC project, a Virtual Kubelet Provider implementation that can be used to bridge Kubernetes and HPC environments.