Proxmox

In an attempt to automate the deployment as well as get my feet wet with a little Ansible in my homelab i created a playbook to automate the deployment of the minimal LXCs that build to deploy certain apps like pi-hole, nginx proxies, and sql databases. The documentation on certain things were pretty sparse so I will try to explain the intentions that I had, the issues that I ran into, as well as why I chose the routes that I took for my solution.

The Proxmox team doesnt really have any plans on changing the default port assigned to Proxmox (8006) and their documentation just tells you to use nginx to proxy the traffic if you want to change the default port so the following script should change your port to the port of your choosing [with 443 as the default].

#!/bin/bash
#Install nginx
apt install nginx

#Checks for your default nginx file and deletes it
FILE=/etc/nginx/conf.d/default
if [-f "FILE"];then
	rm $FILE
else
	rm /etc/nginx/sites-enabled/default
fi

#Pull the FQDN out of the hosts file
read -p "Enter the FQDN of your server or [ENTER] to set to default from /etc/hosts file" FQDN
if [ -z FQDN ]
then FQDN=$(hostname -f)
fi

#Create your new nginx File
cat > /etc/nginx/conf.d/proxmox.conf << EOF
upstream proxmox {
    server "$FQDN";
}

server {
    listen 80 default_server;
    rewrite ^(.*) https://$host$1 permanent;
}

server {
    listen 443;
    server_name _;
    ssl on;
    ssl_certificate /etc/pve/local/pve-ssl.pem;
    ssl_certificate_key /etc/pve/local/pve-ssl.key;
    proxy_redirect off;
    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade"; 
        proxy_pass https://localhost:8006;
	proxy_buffering off;
	client_max_body_size 0;
	proxy_connect_timeout  3600s;
        proxy_read_timeout  3600s;
        proxy_send_timeout  3600s;
        send_timeout  3600s;
    }
}
EOF

#test your config
nginx -t
#reload nginx
nginx -s reload

#creating some dependencies via systemd overrides
cat > /etc/systemd/system/nginx.service.d/override.conf << EOF
[Unit]
Requires=pve-cluster.service
After=pve-cluster.service
EOF

systemctl restart nginx
systemctl enable --now nginx

I don’t really like like allowing servers to set their own IP addresses, i think its kinda weird and i like to handle things at the network level so I typically have all my servers as DHCP clients and I set their addresses statically on the network device they’re attached to. Unfortunately proxmox doesn’t like that so it doesn’t include DHCP in the installer, which is fine. Its easily fixed.

This tutorial is assuming that you already have Tailscale installed on your Proxmox hosts, if you havent done so look at the installation for Tailscale on Debaian Bullseye.

Install#

1. Because /etc/hosts has priority in all host lookups we are just going to edit the these files on any of the machines that we would like to connect. To do so you can simply vim /etc/hosts and your new host files will end up looking like this with host being the node you want to start the cluster on and remote being the node that will join the cluster

   Read more >

Theres no real way to delete a cluster in the gui with out doing a fresh install (that i know of) but the following set of commands should stop the cluster service, force it to local, delete the configs and restart it like new.

systemctl stop pve-cluster
systemctl stop corosync
pmxcfs -l
rm /etc/pve/corosync.conf
rm -r /etc/corosync/*
killall pmxcfs
systemctl start pve-cluster

Introduction#

For those who aren’t familiar with the concept. Different hypervisors use different file formats to represent VM images and due to the fact that Proxmox uses Qemu (Quick Emulator)/KVM (Kernel-based Virtual Machine) to provide virtualization, it in turn uses Qcow2 (Qemu Copy-On-Write) as the storage file format for virtual machines data. The process to move Qcow2 files into Proxmox may not be as straight forward as it is on VMware, but its still a relatively painless process.