Detailed Summary
A robust Kubernetes cluster begins with a reproducible infrastructure layer. Rather than manually clicking through the Proxmox GUI, this pipeline leverages Terraform to define the node topology declaratively, and kubeadm (Ansible) to set up a cluster with 3 control plane nodes and 3 worker nodes. You can find the code at the end of each section.
1. Creating Proxmox VMs with Terraform
Instead of clicking through Proxmox Web Interface, I use Terraform which handles the heavy lifting of interacting with the Proxmox API. Its responsibilities include:
- Resource Allocation: Creating the virtual machines across the Proxmox environment (CPU, memory, disk, and network bridging).
- Environment Bootstrapping: Generating and attaching dynamic configurations for the first boot.
- Agent Integration: Installing baseline packages, such as
qemu-guest-agent, ensuring the Proxmox hypervisor maintains deep visibility into the VM's status and IP allocation.
The following section describe this process in great detail
Structuring the Infrastructure Code
To keep the deployment maintainable and easy to read, I avoided dumping everything into a single monolithic main.tf file. Instead, the Terraform configuration is logically divided by responsibility:
├── provider.tf # Configures the Proxmox bpg provider and API credentials
├── variables.tf # Defines inputs (VM counts, resource sizes, datastores)
├── vms.tf # The core logic for provisioning the QEMU virtual machines
├── cloudconfig.tf # Generates dynamic cloud-init templates for bootstrapping
└── .gitignore # Ensures secrets and state files stay out of Git
provider.tf: Sets up the connection to the Proxmox cluster using the BPG Proxmox provider.variables.tf: Acts as the interface for the module. By extracting values like node counts, RAM, and CPU cores into variables, the cluster topology can be adjusted without modifying the core logic. (Note: The actual secrets and specific values are injected locally via a terraform.tfvars file).vms.tf: Contains the actual resource definitions (proxmox_virtual_environment_vm). It loops through our variable definitions to stamp out the exact number of control plane and worker nodes required.cloudconfig.tf: Handles the zero-touch provisioning aspect by generating the YAML files required forcloud-init, dynamically inserting hostnames and SSH keys for each newly created node.
This modular approach makes the codebase much easier to digest, troubleshoot, and scale. If I need to change how the OS boots, I only check cloudconfig.tf; if I need to add more RAM to the worker nodes, I simply update my .tfvars.
In my version, these VMs are running Debian 12 (Bookworm) latest version. I see tutorials online using Talos Linux as a new, more efficient OS for creating a Kubernetes cluster from scratch, but for this simple homelab environment, I choose Debian 12 and for future projects, I might as well use Talos Linux.
Again, the source code for this can be found here.
2. Install Kubernetes with Kubeadm
2.2. Load balancer for kubeapi-server
For a highly available kubeadm cluster, every control plane node and worker node needs a stable API endpoint. If the cluster is initialized against a single control plane IP, that node becomes a hidden single point of failure. The usual answer is to put a load balancer in front of the kube-apiserver instances and use that virtual address as the controlPlaneEndpoint.
At first, I considered the common homelab setup: Keepalived for a floating virtual IP and HAProxy for forwarding traffic to the kube-apiserver processes. That pattern works well when the load balancer nodes are placed across independent failure domains. In my case, though, the topology is smaller:
- 6 VMs total.
- 3 control plane nodes.
- 3 worker nodes.
- 2 physical machines.
- 2 control plane VMs on one physical machine, and 1 control plane VM on the other.
With that layout, adding separate Keepalived and HAProxy VMs does not really solve the important failure case. If the physical machine that hosts two control plane nodes goes down, the cluster loses two out of three etcd members. At that point etcd no longer has quorum, so the Kubernetes API cannot function correctly no matter how healthy the load balancer is. If the other physical machine goes down, the cluster still has two control plane nodes and can continue. That means the result depends heavily on which physical host fails.
Because of that, I chose kube-vip instead of building a separate Keepalived + HAProxy layer. The goal here is not to magically make a two-host lab behave like a three-failure-domain production cluster. The goal is to keep the API endpoint simple, reproducible, and close to the Kubernetes control plane itself.
kube-vip runs as a static pod on the control plane nodes and advertises a virtual IP for the Kubernetes API server. Only one node owns the VIP at a time. If that node becomes unavailable, another eligible control plane node takes over the VIP, and clients can continue using the same API endpoint.
The kubeadm configuration then points controlPlaneEndpoint at the kube-vip address:
controlPlaneEndpoint: "192.168.1.50:6443"
From the rest of the cluster's perspective, this gives a single stable endpoint:
https://192.168.1.50:6443
The practical tradeoff is:
kube-vipgives me a clean highly available API address without running extra load balancer VMs.- It reduces moving parts compared with Keepalived + HAProxy.
- It fits well with kubeadm because it can be deployed as a static pod before the first control plane is fully initialized.
- It does not fix the underlying physical-host failure-domain issue.
For this homelab, that tradeoff is acceptable. The cluster is still limited by having only two physical machines, but kube-vip keeps the Kubernetes API endpoint HA within the constraints of the hardware I actually have. A more correct production-style design would place the three control plane nodes across three independent physical machines, or use an external load balancer that also lives outside the Kubernetes failure domain.