5 Proxmox Kubernetes Configurations You Need to Know

As more people adopt Proxmox for their home labs or even production environments, you might be curious about the options for running Kubernetes on it. How can Kubernetes be set up with Proxmox? There are several ways to configure a Kubernetes deployment in Proxmox. Let’s explore the key Proxmox Kubernetes configurations and their architectures.

Proxmox is a great platform for K8s

With Proxmox emerging as a popular hypervisor choice, many users interested in experimenting with or running Kubernetes are exploring the best approaches to achieve this on the platform. Fortunately, Proxmox offers several options.

You can leverage its built-in LXC container functionality, take the traditional virtual machine route, or even run Kubernetes in containers using K3D. With so many methods available, I encourage you to explore and experiment with the different configurations to find the one that best suits your needs.

Proxmox Kubernetes configurations

We will consider the following Proxmox Kubernetes configurations that provide great options for home lab or production environments:

1. Traditional Kubernetes virtual machine nodes
2. Talos Linux Kubernetes
3. Kubernetes in LXC containers
4. K3D
5. Run on Proxmox bare metal

1. Traditional Kubernetes virtual machine nodes

This is the simplest deployment approach, where you create three traditional Linux virtual machines and install your preferred Kubernetes distribution on them. Recently, I’ve been opting for Ubuntu Server virtual machines with Microk8s installed.

Microk8s is a production-ready Kubernetes distribution that is straightforward to install and manage. Its snap package integrates all necessary commands, making it easy to enable or disable components within your Microk8s cluster.

Microk8s seamlessly integrates with Microceph, enabling efficient Kubernetes storage management using the rook-ceph components specifically designed for Microk8s.

Pros

• Very easy and traditional way to configure K8s
• Backups of your Kubernetes nodes are easy since they are traditional VMs

Cons

• Takes more resources
• Management of virtual machine lifecycle, updates, etc

2. Talos Linux Kubernetes

Another Kubernetes option that is rapidly gaining popularity is Talos Linux Kubernetes. Talos Linux is a highly streamlined distribution with a strong emphasis on security. It eliminates the ability to SSH into your Talos Linux hosts, and everything is managed through an API-driven approach.

Talos Linux Kubernetes follows a bootstrapping process to get started, where you first deploy the VMs, then bootstrap the cluster to bring it online. For more information on setting this up in Proxmox, check out my detailed blog post on how to install Proxmox Kubernetes with Talos Linux.

Pros

• Very secure
• API-driven
• No SSH access

Cons

• Steeper learning curve
• May have challenges compared to other Kubernetes distributions

3. Kubernetes in LXC containers

Another excellent option, particularly for home labs, is running Kubernetes in Proxmox. Proxmox has native support for LXC containers, which can be easily created either through the web GUI or the command line. LXC containers offer a middle ground between full virtual machines and Docker containers, providing a versatile solution for containerized environments.

It is a more operating system-like container than Docker which is more application-centric. These are still much more lightweight than a full VM and you can run Kubernetes inside of your LXC containers. This provides a very lightweight and streamlined environment where you can run Kubernetes.

Pros

• Very lightweight
• Easy for lab environments

Cons

• May not be the best for production environments
• Can have limitations

4. K3D

Another option to run Proxmox Kubernetes clusters is using the K3D project. What is K3D? K3D is a utility that serves as a lightweight tool that lets you run your K3s Kubernetes environment inside Docker containers.

Pros

• Easy to spin up
• Great for lab environments
• Lightweight

Cons

• Probably not the best for production
• Steeper learning curve

5. Run on Proxmox bare metal

This option is ideal if you want to run your Kubernetes cluster on bare metal while also supporting a dual-purpose setup for running both VMs and containers. Since Proxmox is built on Debian, you can treat the Proxmox host as a Kubernetes bare metal node and directly install your Kubernetes distribution on the hosts.

Pros

• You have the performance and performance of bare metal
• You have the platform host ready to go by default
• Ok for lab environments

Cons

• Not for production
• Mixes your physical host environment with your virtualization host

Wrapping up

I hope this blog post provides you with some ideas for Proxmox Kubernetes configurations that you can experiment with in your home lab or even in a production environment to discover the best solution for your needs. Each method comes with its own set of advantages and drawbacks, so be sure to consider these when making your choice. Most importantly, make the most of a lab environment before deploying to production. This will allow you to identify and address any issues or limitations with a specific configuration before they arise in a production setting.