Homelab Proxmox + Debian + VyOS upgrade migration

16 Nov, 2024

Here I document my home server config & upgrade path from Debian 11 to 12 / Proxmox 7 to 8. About 2 years ago I migrated to Proxmox (vanwerkhoven.org) as host with clients underneath. So far it’s been a good experience, e.g. migrating Home Assistant from Docker image to separate VM worked and went smoothly. Additionally, when upgrading Debian from 11 to 12 I can create a parallel VM and move services one by one instead of running a full re-install. Finally, I want to upgrade Proxmox itself from 7 to 8, which is the most tricky and might require a re-install.

Setup overview ¶

Goal and hardware are unchanged versus my original post (vanwerkhoven.org).

Target services & architecture ¶

My setup is as follows:

Proxmox (32GB storage + 2GB RAM)
- sharing common bulk storage to guests via mount points
- N.B. I originally had 8GB here which is a bit too tight
VyOS VM (8GB storage + 2GB RAM)
- dns adblock (vanwerkhoven.org)
- wireguard
- regular router config
- fq-codel QoS
- N.B. This has been working stable for a while now. VyOS is feature rich, and well suited to run as VM under Proxmox
Debian Stable LXC ‘unifi’ (8GB thinvol + 2GB RAM)
- unifi-controller installed natively
Debian Stable LXC ‘proteus’ (256GB thinvol + 24GB RAM)
- First prio services
  - Nginx (for website & reverse proxies)
  - Letsencrypt/Lego (for SSL certificates)
  - Docker
    - Nextcloud (for file sharing)
    - bpatrik/pigallery2 (for personal photo sharing)
  - Home automation worker scripts (for data generation/collection)
    - many
  - Influxdb (for data storage)
  - Mosquitto (glueing home automation)
- Second prio
  - Grafana (for data visualization)
  - Plex/Jellyfin/Emby (HTPC)
  - Collectd (for data generation/collection)
  - Proxmox external metrics server
  - smbd (for Time Machine backups)
  - Transmission (downloading torrents)
Home Assistant VM (for monitoring)

Debian migration approach ¶

Plan ¶

Debian Stable LXC ‘proteus’ (256GB thinvol + 24GB RAM)
- High impact (cannot run in parallel)
  - Nginx (for website & reverse proxies)
  - Letsencrypt (for SSL certificates)
  - Docker
    - Nextcloud (for file sharing) - I only use Nextcloud
    - bpatrik/pigallery2 (for personal photo sharing)
  - Home automation worker scripts (for data generation/collection)
    - many
  - Influxdb (for data storage)
  - Mosquitto (glueing home automation)
- Low impact (can run in parallel)
  - Grafana (for data visualization)
  - Plex/Jellyfin/Emby (HTPC)
  - Collectd (for data generation/collection)
  - Proxmox external metrics server
  - Transmission (downloading torrents)

Execution ¶

Build new Debian LXC - OK
Install nginx & lego - OK 1. Migrate nextcloud docker image - OK
1. Migrate pigallery2 docker image - OK 2. Migrate influxDB - OK
2. Migrate web plot script - OK
3. Migrate web plot directory - OK
4. Migrate mqtt2influxdb - OK
5. Migrate mosquitto - OK 4. Update all esphome mqtt pushers - OK, already via hostname
6. Migrate worker scripts pushing to influxdb
7. smeter - OK retired
8. water_meter - OK
9. co2signal - OK
10. knmi - OK
11. epexspot - OK 3. zigbee2mqtt - OK 4. smokeping - OK 3. Grafana - partial 3. Redirect port forward to new nginx
Install dyndns workers - OK 1. TransIP - OK 2. Gandi - OK

Build new Debian LXC ¶

Get image and start LXC ¶

Get images using Proxmox’ Proxmox VE Appliance Manager (proxmox.com):

sudo pveam update
sudo pveam available
# sudo pveam download local debian-11-standard_11.6-1_amd64.tar.zst
sudo pveam download local debian-12-standard_12.7-1_amd64.tar.zst
sudo pveam list local

Check storage to use

pvesm status

Create and configure LXC container (proxmox.com) based on downloaded image. Ensure it’s an unprivileged container to protect our host and router running on it.

sudo pct create 203 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst --description "Debian 12 LXC" --hostname proteus2 --rootfs thinpool_vms:256 --unprivileged 1 --cores 4 --memory 16384 --ssh-public-keys /root/.ssh/tim.id_rsa.pub --net0 name=eth0,bridge=vmbr0,firewall=0,gw=172.17.10.1,ip=172.17.10.7/24,tag=10

Now configure networking, on Proxmox’ vmbr0 with VLAN ID 10. This means the guest can only access VLAN 10.

# This does not work, cannot create network device on vmbr0.10
# pct set 203 --net0 name=eth0,bridge=vmbr0.10,firewall=0,gw=172.19.10.1,ip=172.19.10.2/24
# Does not work:
# pct set 203 --net0 name=eth0,bridge=vmbr0,firewall=0,gw=172.17.10.1,ip=172.17.10.2/24,trunks=10
# Works:
# pct set 203 --net0 name=eth0,bridge=vmbr0,firewall=0,gw=172.17.10.1,ip=172.17.10.2/24,tag=10
sudo pct set 203 --onboot 1

Optional: only required if host does not have this set up correctly (could be because network was not available at init)

sudo pct set 203 --searchdomain lan.vanwerkhoven.org --nameserver 172.17.10.1

If SSH into guest fails or takes a long time, this can be due to LXC / Apparmor security features (stackoverflow.com) which prevent mount from executing. To solve, ensure nesting is allowed (ostechnix.com):

sudo pct set 203 --features nesting=1

To enable Docker (jlu5.com) inside the LXC container, we need both nesting & keyctl:

sudo pct set 203 --features nesting=1,keyctl=1

Initial Debian config ¶

Start & log in, set root password, configure some basics

sudo pct start 203
sudo pct enter 203

passwd
apt install sudo vim

cat << 'EOF' | sudo tee -a /usr/share/vim/vim??/defaults.vim
" TvW 20230808 enable copy-paste - see https://vi.stackexchange.com/questions/13099/not-able-to-copy-from-terminal-when-using-vim-from-homebrew-on-macos
set mouse=r
EOF

dpkg-reconfigure locales
dpkg-reconfigure tzdata

Tweak bashrc to merge history (askubuntu.com) and keep it for longer:

cat << 'EOF' >> ~tim/.bashrc
# TvW 20230812 expand history, add date/time (iso fmt), ignore space/duplicates
HISTSIZE=500000
HISTFILESIZE=1000000
HISTTIMEFORMAT="%F %T "
HISTCONTROL=ignoreboth:erasedups
PROMPT_COMMAND="history -a"
EOF

Add regular user, add to system groups (debian.org), and set ssh key

adduser tim
usermod -aG adm,render,sudo,staff,ssl-cert tim
mkdir -p ~tim/.ssh/
touch ~tim/.ssh/authorized_keys
chown -R tim:tim ~tim/.ssh

cp /root/.ssh/authorized_keys ~tim/.ssh/authorized_keys
chmod og-rwx ~tim/.ssh/authorized_keys

cat << 'EOF' >>~tim/.ssh/authorized_keys
ssh-rsa AAAA...
EOF

# Allow non-root to use ping
setcap cap_net_raw+p $(which ping)

Update & upgrade and install automatic updates (linode.com)

sudo apt update
sudo apt upgrade

sudo apt install unattended-upgrades
# Comment 'label=Debian' to not auto-update too much
sudo vi /etc/apt/apt.conf.d/50unattended-upgrades

# Tweak some settings
cat << 'EOF' | sudo tee -a /etc/apt/apt.conf.d/50unattended-upgrades
Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Remove-New-Unused-Dependencies "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
EOF

sudo unattended-upgrades --dry-run --debug

Enable SSH & firewall resolution

set system static-host-mapping host-name proteus2.lan.vanwerkhoven.org inet 172.17.10.7

set firewall name FW_TRUST2INFRA rule 212 action accept
# set firewall name FW_TRUST2INFRA rule 212 description 'accept mqtt(s)/http(s)/HA/ssh/grafana/jellyfin&emby/plex/iperf/transmission to proteus'
set firewall name FW_TRUST2INFRA rule 212 description 'accept ssh,http(s) to proteus2'
set firewall name FW_TRUST2INFRA rule 212 destination address 172.17.10.7
set firewall name FW_TRUST2INFRA rule 212 protocol tcp
# set firewall name FW_TRUST2INFRA rule 212 destination port 8883,1883,80,443,8123,22,3000,8096,32400,32469,7575,9001
set firewall name FW_TRUST2INFRA rule 212 destination port 22,80,443

Harden setup - TODO ¶

Using lynis (github.com)

/usr/sbin/lynis audit system
sudo apt install apt-listbugs needrestart

Harden system services (ruderich.org) by adding security settings, see also https://unix.stackexchange.com/questions/691008/systemd-analyze-does-not-detect-changes-made-by-systemctl-edit` (stackexchange.com)

sudo systemctl edit $service

CapabilityBoundingSet=
KeyringMode=private
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoNewPrivileges=yes
PrivateDevices=yes
PrivateMounts=yes
PrivateNetwork=yes
PrivateTmp=yes
PrivateUsers=yes
ProtectClock=true
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=true
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
ProtectSystem=strict
# Permit AF_UNIX for syslog(3) to help debugging. (Empty setting permits all
# families! A possible workaround would be to blacklist AF_UNIX afterwards.)
RestrictAddressFamilies=
RestrictAddressFamilies=AF_UNIX
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=
SystemCallFilter=@system-service
SystemCallFilter=~@aio @chown @clock @cpu-emulation @debug @keyring @memlock @module @mount @obsolete @privileged @raw-io @reboot @resources @setuid @swap userfaultfd mincore

# Restrict access to potential sensitive data (kernels, config, mount points,
# private keys). The paths will be created if they don't exist and they must
# not be files.
TemporaryFileSystem=/boot:ro /etc/luks:ro /etc/ssh:ro /etc/ssl/private:ro /media:ro /mnt:ro /run:ro /srv:ro /var:ro
# Permit syslog(3) messages to journald
BindReadOnlyPaths=/run/systemd/journal/dev-log

Harden SSH ¶

Test with ssh-audit (ssh-audit.com) and also see this very old guide (stribik.technology).

Re-generate the RSA and ED25519 keys

sudo rm /etc/ssh/ssh_host_*
sudo ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
sudo ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N "" 
echo -e "\nHostKey /etc/ssh/ssh_host_ed25519_key\nHostKey /etc/ssh/ssh_host_rsa_key" | sudo tee -a /etc/ssh/sshd_config

Remove small Diffie-Hellman moduli

awk '$5 >= 3071' /etc/ssh/moduli | sudo tee -a /etc/ssh/moduli.safe
sudo mv /etc/ssh/moduli.safe /etc/ssh/moduli

Restrict supported key exchange, cipher, and MAC algorithms

echo -e "# Restrict key exchange, cipher, and MAC algorithms, as per sshaudit.com\n# hardening guide.\n KexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,gss-curve25519-sha256-,diffie-hellman-group16-sha512,gss-group16-sha512-,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha256\n\nCiphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-gcm@openssh.com,aes128-ctr\n\nMACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com\n\nHostKeyAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256\n\nRequiredRSASize 3072\n\nCASignatureAlgorithms sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256\n\nGSSAPIKexAlgorithms gss-curve25519-sha256-,gss-group16-sha512-\n\nHostbasedAcceptedAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-256\n\nPubkeyAcceptedAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-256\n\n" | sudo tee -a /etc/ssh/sshd_config.d/ssh-audit_hardening.conf

Implement connection rate throttling - I prefer the sshd version to concentrate sshd config to its file only

echo -e "\nPerSourceMaxStartups 1" | sudo tee -a /etc/ssh/sshd_config

Disallow password login via /etc/ssh/sshd_config:

PasswordAuthentication no
ChallengeResponseAuthentication no

Optimize Debian ¶

Prune big packages (cyberciti.biz)

sudo apt install debian-goodies
dpigs -H -n 20

# Manually installed packages
apt list --manual-installed=true

sudo apt install ncdu

Clean docker cache (stackoverflow.com)

sudo docker image prune -a

Install Docker ¶

Install Docker (docker.com). Need to use custom apt repo to get latest version which works inside an unprivileged LXC container (as proposed on the docker forums (docker.com)):

sudo apt remove docker docker-engine docker.io containerd runc docker-compose

sudo apt update

sudo apt install \
   ca-certificates \
   curl \
   gnupg \
   lsb-release

sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Confirm it’s working

sudo docker run hello-world

Migrate services ¶

Nginx ¶

Install Nginx with Lego (github.io) as ACME certificate manager. Unfortunately, Debian silently disables certain DNS provides (debian.org), so also Debain 12 does not support the DNS providers I need. See also (here (github.com) and here (github.com)).

Approach:

Install nginx & lego
Migrate nginx config
Adapt nginx config to lego
Define virtual hosts via host names
Harden nginx

Install nginx & lego ¶

sudo apt install nginx
sudo apt remove lego
# Install manually instead
wget https://github.com/go-acme/lego/releases/download/v4.20.4/lego_v4.20.4_linux_amd64.tar.gz
mkdir -p ~/download/lego_v4.20.4_linux_amd64
tar xvf lego_v4.20.4_linux_amd64.tar.gz -C ~/download/lego_v4.20.4_linux_amd64

Run Lego for all domains once

TRANSIP_ACCOUNT_NAME="twerkhov" TRANSIP_PRIVATE_KEY_PATH="/etc/ssl/private/transipkey.pem" lego --accept-tos --email tim@vanwerkhoven.org --dns transip --domains isboudewijnretired.nl --path=/etc/ssl/lego run
GANDI_API_KEY_FILE=/etc/ssl/private/gandiapikey lego --accept-tos --email tim@vanwerkhoven.org --dns gandi -d '*.vanwerkhoven.org' --path=/etc/ssl/lego renew
GANDIV5_PERSONAL_ACCESS_TOKEN_FILE=/etc/ssl/private/gandipersonalaccesstoken lego --accept-tos --email tim@vanwerkhoven.org --dns gandiv5 -d '*.vanwerkhoven.org' --path=/etc/ssl/lego  run
install -m 600 -o tim -g tim /dev/null /var/log/lego.log

Allow user to restart nginx in /etc/sudoers

visudo
# Allow user tim to reload nginx after certificate renewal
%tim  ALL=NOPASSWD: /sbin/service nginx reload

Install cronjob, add random sleeper to be a good citizen and load-balance, redirect stderr to stdout (cyberciti.biz) and store in log file.

30 01 * * * perl -e 'sleep int(rand(43200))' && TRANSIP_ACCOUNT_NAME="twerkhov" TRANSIP_PRIVATE_KEY_PATH="/etc/ssl/private/transipkey.pem" lego --accept-tos --email tim@vanwerkhoven.org --dns transip --domains isboudewijnretired.nl --path=/etc/ssl/lego renew >>/var/log/lego.log 2>&1 && sudo service nginx reload
35 01 * * *  perl -e 'sleep int(rand(43200))' && GANDIV5_PERSONAL_ACCESS_TOKEN_FILE=/etc/ssl/private/gandipersonalaccesstoken lego --accept-tos --email tim@vanwerkhoven.org --dns gandiv5 -d '*.vanwerkhoven.org' --path=/etc/ssl/lego renew >>/var/log/lego.log 2>&1 && sudo service nginx reload

Install IP change detectors, check every 5min to minimize downtime

install -m 600 -o tim -g tim /dev/null /var/log/livedns.log
install -m 600 -o tim -g tim /dev/null /var/log/livedns-error.log
sudo apt install python3-netifaces

*/5 * * * * python3 /home/tim/workers/gandi-live-dns/src/gandi-live-dns.py 1>> /var/log/livedns.log 2>> /var/log/livedns-error.log
*/5 * * * * /home/tim/workers/transip-live-dns/transip-dynamic-ip.sh >> /var/log/livedns.log 2>> /var/log/livedns-error.log

Migrate nginx config ¶

Move from old server to new server, review configs, test nginx, restart.

sudo systemctl restart nginx.service
sudo nginx -t

Update trusted proxies in e.g. Home Asssistant

  trusted_proxies:
  - 172.17.10.7

Separate internal/external virtual hosts - TODO ¶

Some virtual hosts I want to limit to LAN, while others should be exposed to WAN. Besides setting allow/deny directives per virtual host, it’s possible to get multiple IPs on a NIC, and bind nginx virtual hosts to seperate IPs.

First, get multiple IPs (cyberciti.biz) for this host. Note that IP aliasing (kernel.org) is deprecated, and one should use the ip tool instead of ifconfig-based solutions.

# in /etc/network/interfaces:
iface eth0 inet static
        address 172.17.10.7/24
        gateway 172.17.10.1
        up   ip addr add 172.17.10.8/24 dev eth0 label eth0:0
        down ip addr del 172.17.10.8/24 dev eth0 label eth0:0

sudo systemctl restart networking

Then, bind to separate IPs, and set up your router/firewall to allow only one IP reachable by WAN.

Review full config ¶

Nginx config can be a bit opaque with various include directives, hence you can dump your nginx config (stackoverflow.com) to review it fully:

sudo nginx -T > nginx-full.conf

Harden Nginx ¶

Sources:

Fix DH to prevent Logjam, use 4096bits to get 100% score in SSL Labs SSL Server Rating (github.com)

openssl dhparam -out ssl-dhparams-weakdh.org-4096.pem 4096

Additionally, this requires some tweaking of the Letsencrypt certificate and pruning 128bit ciphers which I didn’t do.

Grafana ¶

We can either use apt or the docker image. I go for apt here so I can more easily re-use my letsencrypt certificate via /etc/grafana/grafana.ini (grafana.com).

Install for Debian (grafana.com)

sudo apt-get install -y apt-transport-https software-properties-common wget
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

Add repo

echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

Install

sudo apt-get update
sudo apt-get install grafana

Start now & start automatically

sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl status grafana-server
sudo systemctl enable grafana-server.service

Set up letsencrypt HTTPS (1/2) ¶

Enable HTTPS using letsencrypt certificate (grafana.com)

sudo ln -s /etc/letsencrypt/live/vanwerkhoven.org/privkey.pem /etc/grafana/grafana.key
sudo ln -s /etc/letsencrypt/live/vanwerkhoven.org/fullchain.pem /etc/grafana/grafana.crt

# Allow access
sudo groupadd letsencrypt-cert
sudo usermod --append --groups letsencrypt-cert grafana

sudo chgrp -R letsencrypt-cert /etc/letsencrypt/*
sudo chmod -R g+rx /etc/letsencrypt/*
sudo chgrp -R grafana /etc/grafana/grafana.crt /etc/grafana/grafana.key
sudo chmod 400 /etc/grafana/grafana.crt /etc/grafana/grafana.key

Set up HTTPS proxy (2/2) ¶

Use nginx as proxuy for Grafana, so we have SSL maintenance in one place + only ever use nginx to expose to outside world.

server {
  listen 443 ssl http2;
  listen [::]:443 ssl http2;

  server_name grafana.vanwerkhoven.org;

  location / {
    include snippets/nginx-server-proxy-tim.conf;
      # TvW 20241126: only allow from LAN (and thus also via VPN)
      allow 127.0.0.1;
      allow 172.17.0.0/16;
      deny all;

    #client_max_body_size 16G;
    proxy_buffering off;
    #proxy_pass http://grafana.lan.vanwerkhoven.org:3000;
    # Use fixed IP instead because DNS might not be up yet
    # resuting in error
    # "nginx: [emerg] host not found in upstream"
    #proxy_pass http://172.17.10.2:3000;
    # https://stackoverflow.com/questions/32845674/nginx-how-to-not-exit-if-host-not-found-in-upstream
    resolver 172.17.10.1 valid=30s;
    set $upstream_ha grafana.lan.vanwerkhoven.org;
    proxy_pass http://$upstream_ha:3000;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection “upgrade”;
  }
  include snippets/nginx-server-ssl-tim.conf;
  include snippets/nginx-server-cert-vanwerkhoven-tim.conf;
}

Migrate config ¶

Migrate configuration

Install used plugin on new server (none)
Stop Grafana service on source and destination server
Copy /var/lib/grafana/grafana.db from old to new server
Check /etc/grafana/grafana.ini
Reconnect to datasource

sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/grafana/grafana.db tim@proteus2:migrate/grafana.db-migrate
sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /etc/grafana/grafana.ini tim@proteus2:migrate/grafana.ini-migrate

sudo diff grafana.ini-migrate /etc/grafana/grafana.ini # manually port changes in case new version has new syntax
sudo chown grafana:grafana grafana.db-migrate
sudo cp grafana.db-migrate /var/lib/grafana/grafana.db

sudo systemctl enable grafana-server.service
sudo systemctl start grafana-server.service

Set up notifications - TODO ¶

TODO: Set up notifications for everything https://grafana.com/docs/grafana/latest/alerting/fundamentals/alert-rules/message-templating/ (grafana.com) https://grafana.com/docs/grafana/latest/alerting/manage-notifications/template-notifications/using-go-templating-language/ (grafana.com)

Docker ¶

Docker should be easy to migrate from one to another. There’s two things to migrate:

Migrate volumes:
https://docs.docker.com/engine/storage/volumes/#back-up-restore-or-migrate-data-volumes (docker.com)
https://stackoverflow.com/questions/45714456/how-to-migrate-docker-volume-between-hosts (stackoverflow.com)
Migrate containers: can be done but not needed(?)

Nextcloud ¶

Update: alternatively, Sandstorm (github.com) looks good, with security-first mindset.

This guide helps to migrate Nextcloud (rair.dev) from Docker to Docker:

sudo docker exec -u www-data docker-app-1 php occ maintenance:mode --on
sudo docker exec -u mysql docker-db-1 mkdir -m 750 /var/lib/mysql/backup
sudo docker exec -u mysql docker-db-1 bash -c 'umask 007 && mysqldump --single-transaction -u nextcloud \
-p$MYSQL_PASSWORD nextcloud > /var/lib/mysql/backup/nextcloud-sqlbkp_`date +"%Y%m%d"`.bak'
sudo docker exec -u mysql docker-db-1 ls -lh /var/lib/mysql/backup

Backup data

tar cvf docker_nextcloud.tar /var/lib/docker/volumes/docker_nextcloud/_data/
cp /var/lib/docker/volumes/docker_db/_data/backup/*

Restore data & database

tar xvf docker_nextcloud.tar
sudo rsync -Aax --progress var/lib/docker/volumes/docker_nextcloud/_data/data/ /var/lib/docker/volumes/docker_nextcloud/_data/data/
sudo rsync -Aax --progress var/lib/docker/volumes/docker_nextcloud/ /var/lib/docker/volumes/docker_nextcloud/

# Beware of the lack of the leading slash! This is a var subdirectory, not /var!
rm docker_nextcloud.tar && rm -r var/lib/docker/volumes

Alternatively, directly rsync from one machine to another as root (danger!):

sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/docker/volumes/docker_nextcloud/ root@proteus2:/var/lib/docker/volumes/docker_nextcloud/

Restore database from backup, first copy the file to the container volume, then restore to the database.

sudo docker exec -u mysql docker-db-1 mkdir -m 750 /var/lib/mysql/backup
sudo cp nextcloud-sqlbkp_20241127.bak /var/lib/docker/volumes/docker_db/_data/backup/

sudo docker exec docker-db-1 bash -c 'mysql -u nextcloud -p$MYSQL_PASSWORD \
-e "DROP DATABASE nextcloud"'
sudo docker exec docker-db-1 bash -c 'mysql -u nextcloud -p$MYSQL_PASSWORD \
-e "CREATE DATABASE nextcloud"'
sudo docker exec docker-db-1 bash -c 'mysql -u nextcloud -p$MYSQL_PASSWORD \
nextcloud < /var/lib/mysql/backup/nextcloud-sqlbkp_20241127.bak'

Wrap up and restore nextcloud:

sudo docker exec -u www-data docker-app-1 php occ maintenance:mode --off
sudo docker exec -u www-data docker-app-1 php occ maintenance:data-fingerprint
sudo docker exec -u www-data docker-app-1 php occ files:scan --all

Update firewall / DNS

set system static-host-mapping host-name nextcloud.vanwerkhoven.org inet 172.17.10.2
delete system static-host-mapping host-name nextcloud.lan.vanwerkhoven.org
set system static-host-mapping host-name nextcloud.lan.vanwerkhoven.org inet 172.17.10.7

Confirm /var/lib/docker/volumes/docker_nextcloud/_data/config/config.php is still correct (e.g. overwrite_host/overwrite.cli.url).

Ensure your browser’s DNS cache is refreshed. This can take a while (30-60min), even after clicking clear DNS cache (), surprisingly.

If your Nextcloud instance is available from the Internet, you can use the Nextcloud Security Scan (nextcloud.com) and SSL Labs Test (ssllabs.com) to get some recommendations.

sudo vim /var/lib/docker/volumes/docker_nextcloud/_data/config/config.php

  'allowed_admin_ranges' => [
    '127.0.0.1/8',
    '172.17.0.0/16',
    'fd00::/8',
  ],
  'debug' => false,

sudo docker restart docker-app-1

Pigallery2 ¶

Copy over pigallery config (not tmp/ because I wanted to re-generate thumbnails at lower quality to save space)

sudo mkdir -p /var/lib/pigallery2/config/
sudo mkdir -p /var/lib/pigallery2/tmp/
sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/pigallery/config/ root@proteus2:/var/lib/pigallery2/config/

Then start a new pigallery container with this config:

sudo docker compose -f pigallery2-compose.yml up -d

Copy over database (or not?). Might have a different schema between origin and destination. Let’s try anyway. Only copy sqlite.db, not sqlite.db-shm etc.

sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/docker/volumes/docker_db-data/_data/sqlite.db root@proteus2:/var/lib/docker/volumes/docker_db-data/_data/sqlite.db

Seems to work. Next time update the source Docker host before migrating database to prevent possibly schema mismatch.

Bonus: check disk usage of album & adapt if needed –> reduce quality to

Mosquitto ¶

Install daemon and clients

sudo apt install mosquitto mosquitto-clients

Port configuration, don’t use SSL for now

cat << 'EOF' | sudo tee  /etc/mosquitto/conf.d/tim.conf
# TvW 20190818
# From https://www.digitalocean.com/community/questions/how-to-setup-a-mosquitto-mqtt-server-and-receive-data-from-owntracks
connection_messages true
log_timestamp true

# https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-the-mosquitto-mqtt-messaging-broker-on-ubuntu-16-04
# TvW 201908
allow_anonymous false
password_file /etc/mosquitto/passwd

listener 1883
EOF

cat << 'EOF' | sudo tee /etc/mosquitto/conf.d/ssl-tim.conf.off
# Letsencrypt needs different CA https://mosquitto.org/blog/2015/12/using-lets-encrypt-certificates-with-mosquitto/ 
# Or not?
#cafile /etc/ssl/certs/DST_Root_CA_X3.pem
certfile /etc/letsencrypt/live/vanwerkhoven.org/cert.pem
cafile /etc/letsencrypt/live/vanwerkhoven.org/chain.pem
keyfile /etc/letsencrypt/live/vanwerkhoven.org/privkey.pem

tls_version tlsv1.2
listener 8883
EOF

Create test user & port users from old server

sudo install -m 600 -o mosquitto -g tim /dev/null /etc/mosquitto/passwd
sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /etc/mosquitto/passwd root@proteus2:/etc/mosquitto/passwd

Test run config (sudo is important, else you might get Error: Unable to write pid file.)

sudo /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf -v

If you get a PID error, adapt your mosquitto.conf:

# TvW 20230715 Gave an error? https://github.com/eclipse/mosquitto/issues/1950
#pid_file /run/mosquitto/mosquitto.pid
pid_file /var/run/mosquitto/mosquitto.pid

Optional: if running mosquitto >2.0 and using letsencrypt certificates, ensure to copy them properly after deployment (mosquitto.org) using e.g. this script (github.com). I’m not using this as it requires too many moving parts. Instead, consider using a 100-years self-signed cert.

Go live: change DNS, restart server

mqtt2influxdb ¶

Install (stackexchange.com) as systemd service (stackoverflow.com):

sudo apt install python3-paho-mqtt
sudo cp mqtt2influxdb.service  /etc/systemd/system/
sudo systemctl enable mqtt2influxdb.service
sudo systemctl start mqtt2influxdb.service

Check everything went well

sudo systemctl start mqtt2influxdb.service
journalctl -u mqtt2influxdb.service

InfluxDB ¶

While I prefer to use Debian native packages, it’s old for no reason, even by Debian standards (debian.org) (" The package is severely out of date with respect to the Debian Policy."). Hence here I opted for installing from the InfluxDB repo itself (influxdata.com). I changed the signature location to /etc/apt/keyrings because docker and grafana were already there.

wget -q https://repos.influxdata.com/influxdata-archive_compat.key
echo '393e8779c89ac8d958f81f942f9ad7fb82a25e133faddaf92e15b16e6ac9ce4c influxdata-archive_compat.key' | sha256sum -c && cat influxdata-archive_compat.key | gpg --dearmor | sudo tee /etc/apt/keyrings/influxdata-archive_compat.gpg > /dev/null
echo 'deb [signed-by=/etc/apt/keyrings/influxdata-archive_compat.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list

Now install influxdb:

sudo apt-get update 
sudo apt-get install influxdb
sudo systemctl unmask influxdb.service
sudo systemctl start influxdb

Migrate config, reload

scp -P 10022 tim@172.17.10.107:/etc/influxdb/influxdb.conf influxdb.conf-migrate
sudo diff /etc/influxdb/influxdb.conf /etc/influxdb/influxdb.conf-migrate
sudo diff -I '#.*' -I '  #.*' -I ' #.*' influxdb.conf /etc/influxdb/influxdb.conf 
sudo service influxdb restart
journalctl -u influxdb.service

For collectd to work, copy over types.db from old system (alternatively, install collectd). Ensure the influxdb config file points to the right path.

scp /usr/share/collectd/types.db proteus2:
sudo mkdir -p  /usr/share/collectd
mv types.db /usr/share/collectd/types.db

Make backup on old system, restore on new system


/usr/bin/influxd backup -portable /home/tim/backup/influx_snapshot.db/$(date +%Y%m%d)-migrate
sudo systemctl stop influxdb.service
scp -r /home/tim/backup/influx_snapshot.db/$(date +%Y%m%d)-migrate proteus2:migrate/

/usr/bin/influxd restore -portable /home/tim/migrate/$(date +%Y%m%d)-migrate

Add users in InfluxDB (influxdata.com)

influx -precision rfc3339

CREATE USER influxadmin WITH PASSWORD 'pwd' WITH ALL PRIVILEGES
CREATE USER influxwrite WITH PASSWORD 'pwd'
GRANT WRITE ON collectd TO influxwrite
GRANT WRITE ON smarthomev3 TO influxwrite
CREATE USER influxread WITH PASSWORD 'pwd'
GRANT READ ON collectd TO influxread
GRANT READ ON smarthomev3 TO influxread
CREATE USER influxreadwrite WITH PASSWORD 'pwd'
GRANT ALL ON collectd TO influxreadwrite
GRANT ALL ON smarthomev3 TO influxreadwrite

Test account with curl

chmod o-r ~/.profile
cat << 'EOF' >>~/.profile
export INFLUX_USERNAME=influxadmin
export INFLUX_PASSWORD=pwd
EOF

curl -G http://localhost:8086/query --data-urlencode "q=SHOW DATABASES"
curl -G http://localhost:8086/query -u influxwrite:pwd   --data-urlencode "q=SHOW DATABASES"
influx -precision rfc3339 -database smarthomev3

In case InfluxDB is not running, check that path to types.db is correct.

Failed to connect to http://localhost:8086: Get "http://localhost:8086/ping": dial tcp [::1]:8086: connect: connection refused
Please check your connection settings and ensure 'influxd' is running.

Update DNS

set system static-host-mapping host-name mqtt.lan.vanwerkhoven.org inet 172.17.10.7
set system static-host-mapping host-name influxdb.lan.vanwerkhoven.org inet 172.17.10.7

Check all metrics are coming in:

From Home Assistant –> yes nibe coming in
From zigbee2mqtt –> always via HA
From kaifa –> via HA

Retire on old system

sudo systemctl disable influxdb.service

Worker scripts ¶

First clean up versions, sync between server and repository (I wasn’t so tidy in keeping versions on my server in sync).

mkwebdata ¶

sudo install -m 660 -o tim -g www-data -d /var/www/html/smarthome-197da5
sudo install -m 660 -o tim -g www-data -d /var/www/html/smarthome-9084e7
sudo install -m 660 -o tim -g www-data -d /var/www/www
sudo install -m 660 -o tim -g www-data -d /var/www/isboudewijnretired.nl
rsync -avzAXH --exclude=".git/" /var/www/ proteus2:/var/www/

## Oops, python3-pandas is a 1,599MB / 222 package dependency... Next time try without pandas?
sudo apt install jq python3-pandas python3-influxdb

Test-run

/home/tim/workers/mkwebdata/influx2web.py --outputdir /var/www/html/smarthome-9084e7/data/
/home/tim/workers/mkwebdata/mkwebdata_minutely-knaus.sh

epexspot ¶

Install epexspot in Python venv with own pip dependencies (entsoe-py & pyyaml).

python3 -m venv epex-venv
epex-venv/bin/pip install entsoe-py pyyaml
source /home/tim/workers/epexspot/epex-venv/bin/activate && python3 /home/tim/workers/epexspot/epexspot2influx2b.py && deactivate

Install crontab

knmi ¶

Install dependencies & crontab

sudo apt install python3-netcdf4

# Get verified historical KNMI data daily around noon (but +-1hr), this is when data from previous day becomes available
0 12 * * * /home/tim/workers/knmi2influxdb/knmi2influxdb-wrapper_historical.sh
# Get real-time KNMI data hourly at 10min past
10 * * * * /home/tim/workers/knmi2influxdb/knmi2influxdb-wrapper_actual.sh

smeter / dsmr ¶

Two options to choose from:

https://dsmr-reader.readthedocs.io/en/latest/ (readthedocs.io) + mqtt push to influxdb or
Pro: dedicated interface
Con: extra setup to maintain
Home Assistant Plugin + automation push to mqtt to influxdb
Pro: only one setup
Con: depends on HA

I chose the second because of less moving parts.

Stop current smeter cronjob - OK
Add USB to VM - OK via PVE web interface
Reboot VM - OK
Install DSMR integration (home-assistant.io), using DSMR v5 - OK
Automate values to influxdb via mqtt - OK
Dismantle USB mount in proteus LXC - OK
Reboot cluster to ensure hardware works - OK

co2signal ¶

Cron job

45 * * * * /home/tim/workers/co2signal2influxdb/co2signal2influxdbv3.sh NL
45 * * * * /home/tim/workers/co2signal2influxdb/co2signal2influxdbv3.sh DE
45 * * * * /home/tim/workers/co2signal2influxdb/co2signal2influxdbv3.sh BE
45 0,12 * * * /home/tim/workers/co2signal2influxdb/co2signal2influxdbv3-history.sh NL
45 0,12 * * * /home/tim/workers/co2signal2influxdb/co2signal2influxdbv3-history.sh DE
45 0,12 * * * /home/tim/workers/co2signal2influxdb/co2signal2influxdbv3-history.sh BE

Automated backups ¶

Grafana ¶

From Grafana docs (grafana.com) when using SQLite.

usermod -aG grafana tim
mkdir -p /home/tim/backup/grafana/
tar -cvzf /home/tim/backup/grafana/grafana_snapshot.tar.gz /var/lib/grafana/ && ln /home/tim/backup/grafana/grafana_snapshot.tar.gz /home/tim/backup/grafana/grafana_snapshot-monthly.tar.gz

InfluxDB ¶

/usr/bin/influxd backup -portable /home/tim/backup/influx/influx_snapshot && tar --remove-files -cvzf /home/tim/backup/influx/influx_snapshot.tar.gz -C /home/tim/backup/influx/ influx_snapshot && ln /home/tim/backup/influx/influx_snapshot.tar.gz /home/tim/backup/influx/influx_snapshot-monthly.tar.gz

cat << 'EOF' | sudo tee /etc/logrotate.d/tim-backups
/home/tim/backup/influx/influx_snapshot.tar.gz {
    su tim tim
    daily
    rotate 5
    copy
    nocompress
    ifempty
    missingok
    nocreate
    extension .tar.gz
}
/home/tim/backup/influx/influx_snapshot-monthly.tar.gz {
    su tim tim
    monthly
    rotate 5
    copy
    nocompress
    ifempty
    missingok
    extension .tar.gz
}
/home/tim/backup/grafana/grafana_snapshot.tar.gz {
    su tim tim
    daily
    rotate 5
    copy
    nocompress
    ifempty
    missingok
    nocreate
    extension .tar.gz
}
/home/tim/backup/grafana/grafana_snapshot-monthly.tar.gz {
    su tim tim
    monthly
    rotate 12
    copy
    nocompress
    ifempty
    missingok
    extension .tar.gz
}
EOF

Home Assistant - TODO ¶

Create daily backups from HAOS, remove old ones. Not sure how to run a cron script in HA OS.

#!/bin/sh

# Register older backup by slug name
OLD_BACKUPS=$(ls /mnt/data/supervisor/backup/)

# Stop Home Assistant
ha core stop

# Create Backup
ha backups new --name "Automated backup $(date +%Y-%m-%d)"

# Start Home Assistant
ha core start

# Remove old backups
for filename in $OLD_BACKUPS; do slug=$(echo $filename | cut -f1 -d.); echo ha backups remove $slug; done

Optional / future: combine with logrotate to create tiered backups (vanwerkhoven.org).

zigbee2mqtt ¶

Install as Home Assistant add-on:

As part of HA it should get backuped
Interface available via HA
Alternatively, installing on Debian directly requires pinned non-apt packages (e.g. nodejs 20.0)

To install, see zigbee2mqtts README (github.com). To migrate, see this chapter (github.com).

First setup SSH access to HAOS itself (home-assistant.io), which we need to migrate our setup:

# Login to VM
qm set 101 -serial0 socket
sudo qm terminal 101
# enter root / no password to enter the system
install -m 600 -o root -g root /dev/null /root/.ssh/authorized_keys
# Paste in SSH key
vi /root/.ssh/authorized_keys
systemctl start dropbear

Stop zigbee2mqtt on original host

sudo systemctl stop zigbee2mqtt.service

Now start the service once with fake tty device to create /mnt/data/supervisor/homeassistant/zigbee2mqtt/, then copy over data. Leave out log files.

scp -P 22222 /opt/zigbee2mqtt/data/* root@homeassistant.lan.vanwerkhoven.org:/mnt/data/supervisor/homeassistant/zigbee2mqtt/

Now attach USB device to HAOS via proxmox web GUI, configure & start zigbee2mqtt, hope that network survives :p

server: mqtt://mqtt.lan.vanwerkhoven.org:1883
user: user
password: pass

Retire original server & prevent from starting again

sudo systemctl disable zigbee2mqtt.service

smokeping ¶

sudo apt install smokeping fcgiwrap

# Review /etc/smokeping/config.d
sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/smokeping/dns root@proteus2:/var/lib/smokeping/

# Update nginx conf
# e.g. https://github.com/vazhnov/smokeping_nginx/blob/main/simple.conf

# Tweak /usr/share/smokeping/www/smokeping.fcgi
#!/bin/sh
#exec /usr/sbin/smokeping_cgi /usr/share/doc/smokeping/config
exec /usr/share/smokeping/smokeping.cgi /etc/smokeping/config

# Migrate history
sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/smokeping/dns root@proteus2:/var/lib/smokeping/
sudo --preserve-env=SSH_AUTH_SOCK rsync -Aax --progress /var/lib/smokeping/network root@proteus2:/var/lib/smokeping/

Media server ¶

Server Options:

Plex: good server, closed source, cannot do HW accel, cannot use without account
Emby: good server, closed source, cannot do HW accel (current)
Jellyfin: good server, open source, can do HW accel –> OK

Client options

Swiftfin - laggy app, NOK
Emby - OK app, episodes organization sometimes confusing, cannot see names / doesn’t have list view (current)
Plex - Probably good, cannot use without
MrMC - Spartan but very functiunal and fast, can connect to plethora of sources, one-time purchase, not developed anymore –> OK
Infuse - OK app, but yearly paid subscription
Kodi - possible by building yourself, but requires laborious process every 7 days OR jailbreak (only available semi-tethered)

Options:

Jellyfin server + MrMC client
Plex server + Plex client
Emby server + Emby client

Jellyfin with MrMC ¶

sudo apt install extrepo
sudo extrepo enable jellyfin

Install YouTube metadata plugin (github.com)

sudo apt-get install yt-dlp

Follow instructions on https://github.com/ankenyr/jellyfin-youtube-metadata-plugin (github.com)

Add repo https://raw.githubusercontent.com/ankenyr/jellyfin-plugin-repo/master/manifest.json
Install plugin YoutubeMetadata
Restart Jellyfin
Ensure filenaming has Youtube ID in square brackets
Add as provider to relevant libraries
Refresh metadata

Get automatic subtitles by registering with OpenSubtitles.com (opensubtitles.com) (succesor to the .org) and setting this account in Jellyfin. Free accounts get 20 subs/day.

Plex ¶

Install as Docker image or via apt source (plex.tv) (I chose apt install because less dependencies)

echo deb https://downloads.plex.tv/repo/deb public main | sudo tee /etc/apt/sources.list.d/plexmediaserver.list
curl https://downloads.plex.tv/plex-keys/PlexSign.key | sudo apt-key add -
sudo apt install plexmediaserver

Disable local network auth (plex.tv) in advanced settings (plex.tv), in my case /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Preferences.xml

<Preferences allowedNetworks="172.17.0.0/255.255.0.0" enableLocalSecurity="False" />

For first login, log in via localhost using ssh tunnel, e.g.

ssh -L 32400:localhost:32400 proteus
open http://localhost:32400/web

Pass through storage ¶

On the host (pve), do:

sudo mkdir /mnt/bulk
sudo chown tim:users /mnt/bulk
sudo chmod g+w /mnt/bulk

Make user on host (bulkdata:bulkdata) that we’ll propagate UID/GID to in the guest:

adduser --home /mnt/bulk --no-create-home --shell /usr/sbin/nologin --disabled-password --uid 1010 bulkdata
adduser --home /mnt/backup/mbp --no-create-home --shell /usr/sbin/nologin --disabled-password --uid 1011 backupmbp
adduser --home /mnt/backup/mba --no-create-home --shell /usr/sbin/nologin --disabled-password --uid 1012 backupmba
adduser --home /mnt/backup/tex --no-create-home --shell /usr/sbin/nologin --disabled-password --uid 1013 backuptex
usermod -aG bulkdata tim

Set up UID/GID mapping to propagate users 1010–1020 to the same uid on the host (e.g. using this tool (github.com)). N.B. this is only required if you want to write from both the host and guest. If you only write in (multiple) guests, you only need to ensure the user/group writing from the different guests have the same UID/GID.

cat << 'EOF' >>/etc/pve/lxc/201.conf
# uid map: from uid 0 map 1010 uids (in the ct) to the range starting 100000 (on the host), so 0..1010 (ct) → 100000..101010 (host)
lxc.idmap = u 0 100000 1010
lxc.idmap = g 0 100000 1010
# we map 10 uids starting from uid 1010 onto 1010, so 1010 → 1010
lxc.idmap = u 1010 1010 10
lxc.idmap = g 1010 1010 10
# we map the rest of 65535 from 1020 upto 101020, so 1020..65535 → 101020..165535
lxc.idmap = u 1020 101020 64516
lxc.idmap = g 1020 101020 64516
EOF

Add the following to /etc/subuid and /etc/subgid (there might already be entries in the file, also for root):

cat << 'EOF' >>/etc/subuid
root:1010:10
EOF
cat << 'EOF' >>/etc/subgid
root:1010:10
EOF

On the client, add user at id 1010:

sudo groupadd -g 1010 bulkdata
sudo useradd bulkdata --uid 1010 --gid 1010 --no-create-home --shell /usr/sbin/nologin
sudo groupmod -aU jellyfin,tim,debian-transmission bulkdata

Now mount the actual bind point

sudo pct shutdown 203
sudo pct set 203 -mp0 /mnt/bulk,mp=/mnt/bulk
sudo pct start 203

Transmission ¶

sudo --preserve-env=SSH_AUTH_SOCK scp settings.json tim@proteus2:migrate/

Ente photos ¶

Sounds nice but I don’t need it

I still like to have nextcloud for easily sharing files & directories (mostly photo albums which Ente could do, but also other files)
Pigallery2 is super simple which I like

From Entes self-hosting docs (ente.io):

git clone --depth 1 --branch photos-v0.9.58 https://github.com/ente-io/ente
mv ente ente-photos-v0.9.58
cd ente-photos-v0.9.58/server
sudo docker compose up --build

sudo apt update 
sudo apt install nodejs npm
sudo npm install -g yarn // to install yarn globally

cd ente/web
git submodule update --init --recursive
yarn install
NEXT_PUBLIC_ENTE_ENDPOINT=http://localhost:8092 yarn dev

#Adblock #Home-Assistant #Networking #Nextcloud #Influxdb #Nginx #Letsencrypt #Security #Server #Smarthome #Debian #Vyos #Proxmox #Unix #Grafana

Homelab Proxmox + Debian + VyOS upgrade migration

Contents

Setup overview ¶

Target services & architecture ¶

Debian migration approach ¶

Plan ¶

Execution ¶

Build new Debian LXC ¶

Get image and start LXC ¶

Initial Debian config ¶

Harden setup - TODO ¶

Harden SSH ¶

Optimize Debian ¶

Install Docker ¶

Migrate services ¶

Nginx ¶

Install nginx & lego ¶

Migrate nginx config ¶

Separate internal/external virtual hosts - TODO ¶

Review full config ¶

Harden Nginx ¶

Grafana ¶

Set up letsencrypt HTTPS (1/2) ¶

Set up HTTPS proxy (2/2) ¶

Migrate config ¶

Set up notifications - TODO ¶

Docker ¶

Nextcloud ¶

Pigallery2 ¶

Mosquitto ¶

mqtt2influxdb ¶

InfluxDB ¶

Worker scripts ¶

mkwebdata ¶

epexspot ¶

knmi ¶

smeter / dsmr ¶

co2signal ¶

Automated backups ¶

Grafana ¶

InfluxDB ¶

Home Assistant - TODO ¶

zigbee2mqtt ¶

smokeping ¶

Media server ¶

Jellyfin with MrMC ¶

Plex ¶

Pass through storage ¶

Transmission ¶

Ente photos ¶