VyOS from scratch with VLANs and Zone Based Firewall

28 Nov, 2024

Here I document my VyOS from scratch setup after moving my home server to Proxmox (vanwerkhoven.org) and virtualizing my router as well. See my previous networking setup here (vanwerkhoven.org).

Goal ¶

Router requirements (from home networking setup (vanwerkhoven.org))

Should support 100 MBit WAN (NAT/firewalling requirement)
Separate networks for internal, guest, and buggy IoT devices (VLAN-aware ethernet and WiFi)
Ability to prevent buffer bloat (need decent QoS)
Gigabit LAN
Low-power (ideally <10W for full setup)
LAN-wide adblocking (DNS-based pi-hole or related)
Home VPN server (IKEv2 or Wireguard, >50Mbps)

Build VyOS LTS ¶

You can only build VyOS stable/LTS (sagitta) if you have a subscription, the repository is blocked since April 2024 (vyos.io). If you do have access, below are steps to build VyOS LTS in a docker container in a Debian VM:

Install Debian VM ¶

# Get ISO from https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/
ls /var/lib/vz/template/iso/
sudo qm create 300 --name vyos-builer --description "Debian VM server for VyOS building" --cores 4 --memory 4096 --net0 virtio,bridge=vmbr0,firewall=0,tag=10 --ide2 media=cdrom,file=local:iso/debian-12.8.0-amd64-netinst.iso --virtio0 thinpool_vms:100
# ipconfig0 did not work? --ipconfig0 gw=172.17.10.1,ip=172.17.10.2/24 
sudo qm set 300 -serial0 socket

Open terminal via Spice/xterm.js, install image, remove image, and reboot

sudo qm start 300

# in guest: install image as usual
sudo qm set 300 --ide2 none
qm reboot 300

Optional: add QEMU guest agent

qm set 300 --agent 1
qm agent 300 ping

Build from Docker ¶

Notes based on VyOS docs (vyos.io).

Get docker (see above), then get VyOS build image (current):

mkdir download
cd download
sudo docker pull vyos/vyos-build:current
git clone -b current --single-branch https://github.com/vyos/vyos-build

cd vyos-build
sudo docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build:current bash

sudo make clean
sudo ./build-vyos-image generic --architecture amd64 --build-by "tim@vanwerkhoven.org"

You can’t build stable/LTS (sagitta) images anymore because the repository is blocked since April 2024 (vyos.io)

sudo docker pull vyos/vyos-build:sagitta
git clone -b sagitta --single-branch https://github.com/vyos/vyos-build vyos-build-sagitta

cd vyos-build-sagitta
sudo docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build:sagitta bash

sudo make clean
sudo ./build-vyos-image iso --architecture amd64 --build-by "tim@vanwerkhoven.org"
[...]
E: Failed to fetch http://dev.packages.vyos.net/repositories/sagitta/dists/sagitta/InRelease  403  Forbidden [IP: x.x.x.x 443]

Troubleshooting ¶

Operation not permitted (vyos.io) happens when I used Docker on a non-privileged LXC. Better to use a VM for this.

P: If the following stage fails, the most likely cause of the problem is with your mirror configuration or a caching proxy.
P: Running debootstrap...
mknod: /vyos/build/chroot/test-dev-null: Operation not permitted
E: Cannot install into target '/vyos/build/chroot' mounted with noexec or nodev

sudo mount -i -o remount,exec,dev /vyos
# Doesn't work

Clean up VM

sudo qm destroy 300

Install VyOS as VM ¶

Get image, install and add serial socket (vyos.io) for using xterm.js support (copy-pasting). Also start on boot.

ls /var/lib/vz/template/iso/
qm create 101 --name vyos --memory 2048 --net0 virtio,bridge=vmbr0 --net1 virtio,bridge=vmbr1 --ide2 media=cdrom,file=local:iso/vyos-rolling-latest.iso --virtio0 thinpool_vms:8
qm set 101 --net1 virtio,bridge=vmbr1
qm set 101 -serial0 socket
qm set 101 --onboot 1

Open terminal via Spice/xterm.js, install image, remove image, and reboot

qm start 101
# log in to guest, run `install image` and follow instructions
qm set 101 --ide2 none
qm reboot 101

Enable QEMU guest agent (proxmox.com) in Proxmox (VyOS has this since 2018). Source (itsfullofstars.de)

qm set 101 --agent 1
qm agent 101 ping

Update VyOS version ¶

You can update the VyOS (vyos.io) version by downloading a new image and installing it:

add system image https://github.com/vyos/vyos-nightly-build/releases/download/1.5-rolling-202411230007/vyos-1.5-rolling-202411230007-generic-amd64.iso
reboot

Unfortunately the rolling releases aren’t signed, only LTS are. There are mythical early production access images (vyos.io) close to LTS which I wasn’t able to find, however.

Upgerading should convert your config automatically. https://forum.vyos.io/t/upgrade-from-1-4-to-1-5/13323 (vyos.io)

Upgrading from 1.4 to 1.5 ¶

From 1.4-rolling-2023* to 1.5.

Create manual VM backup in PVE.
Ensure you can connect to network manually (in case VyOs goes down)- find port in switch, connect, set manual address, connect to pve.
Add new image add system image https://github.com/vyos/vyos-nightly-build/releases/download/1.5-rolling-x/vyos-1.5-rolling-x-generic-amd64.iso

Trying to fetch ISO file from https://github.com/vyos/vyos-nightly-build/releases/download/1.5-rolling-x/vyos-1.5-rolling-x-generic-amd64.iso...
Downloading...
Redirecting to https://objects.githubusercontent.com/github-production-release-asset-xxx/xxx
The file is 493.000 MiB.
[#################################################################] 100%
Download complete.
Done.
Checking for digital signature file...
Downloading...
Redirecting to https://objects.githubusercontent.com/github-production-release-asset-xxx/xxx
The file is 0.334 KiB.
[#################################################################] 100%
Download complete.
Checking digital signature...
Signature key id in /var/tmp/install-image.4315/vyos-1.5-rolling-x-generic-amd64.iso.minisig is xxx
but the key id in the public key is xxx
Signature check FAILED, trying BACKUP key...
Signature key id in /var/tmp/install-image.4315/vyos-1.5-rolling-x-generic-amd64.iso.minisig is xxx
but the key id in the public key is xxx
Digital signature is valid.
Checking SHA256 checksums of files on the ISO image... OK.
Done!
What would you like to name this image? [1.5-rolling-x]: 
OK.  This image will be named: 1.5-rolling-x
Installing "1.5-rolling-x" image.
Copying new release files...
Would you like to save the current configuration 
directory and config file? (Yes/No) [Yes]: Yes
Copying current configuration...
Would you like to save the SSH host keys from your 
current configuration? (Yes/No) [Yes]: 
Copying SSH keys...
Running post-install script...
Setting up grub configuration...
Done.

Save config: configure; save; exit; - This is required because somehow the add system image script triggers config changes?

vyos@vyos:~$ reboot 
Warning: there are unsaved configuration changes!
Run 'save' command if you do not want to lose those changes after reboot/shutdown.
Are you sure you want to reboot this system? [y/N] N
vyos@vyos:~$ save

Connect to terminal via pve to watch debug output on reboot
Reboot reboot
Optional: Pray
Reconnect
Profit!

Addendum: check if adblocking still works.

Configure VyOS ¶

Set global settings

set system domain-name 'lan.vanwerkhoven.org'
set system host-name 'vyos'

Configure eth1 (=vmbr1=WAN on my proxmox host) as WAN connection

For DHCP connection (e.g. when connecting to ISP router)

set interfaces ethernet eth1 address 'dhcp'
set interfaces ethernet eth1 description 'Odido 4G LTE WAN'

For DHCP connection on specific VLAN (e.g. Odido fiber)

set interfaces ethernet eth1 vif 300
set interfaces ethernet eth1 vif 300 description "T-mobile WAN"
set interfaces ethernet eth1 vif 300 address dhcp

For PPPoE connection on specific VLAN (e.g. KPN fiber). N.B. you could set MTU to 1500, but then you need to set your Proxmox WAN interface to a higher value as well.

PPPoE settings for KPN (source (ctfassets.net)):

PPPoE via VLAN 6 (802.1q).
PPPoE authentication PAP with username and password (e.g. internet / internet).
Maximum packet size (MTU) 1500 bytes (rfc4638)
Obtain IPv4 address + DNS servers via PPPoE
Obtain IPv6 address range + DNS servers (IPv6) via DHCPv6-PD request (in PPPoE). Use one address for the router from this range.

set interfaces pppoe pppoe0 authentication password 'internet'
set interfaces pppoe pppoe0 authentication username 'internet'
set interfaces pppoe pppoe0 description 'KPN WAN'
set interfaces pppoe pppoe0 ip adjust-mss 'clamp-mss-to-pmtu'
set interfaces pppoe pppoe0 mtu '1492'
set interfaces pppoe pppoe0 source-interface 'eth1.6'

Set up neworking ¶

Setup VLAN on LAN network, see here (vyos.io)

set interfaces ethernet eth0 description 'LAN'

set interfaces bridge br100 enable-vlan
set interfaces bridge br100 member interface eth0 allowed-vlan '10'
set interfaces bridge br100 member interface eth0 allowed-vlan '20'
set interfaces bridge br100 member interface eth0 allowed-vlan '30'
set interfaces bridge br100 member interface eth0 allowed-vlan '40'
set interfaces bridge br100 stp
set interfaces bridge br100 vif 10 address '172.17.10.1/24'
set interfaces bridge br100 vif 10 description 'VLAN10-Mgmt'
set interfaces bridge br100 vif 20 address '172.17.20.1/24'
set interfaces bridge br100 vif 20 description 'VLAN20-Trusted'
set interfaces bridge br100 vif 30 address '172.17.30.1/24'
set interfaces bridge br100 vif 30 description 'VLAN30-Guest'
set interfaces bridge br100 vif 40 address '172.17.40.1/24'
set interfaces bridge br100 vif 40 description 'VLAN40-IoT'

Enable SSH on only management interface without password auth.

set service ssh port '22'
set service ssh listen-address '172.17.10.1'
set service ssh disable-password-authentication
set system login user vyos authentication public-keys tim@neptune type ssh-rsa
set system login user vyos authentication public-keys tim@neptune key AAAA...

Harden SSH, only allow strong ciphers, don’t use md5/sha1, don’t use contested nistp256 (stackexchange.com):

set service ssh ciphers 'aes128-cbc'
set service ssh ciphers 'chacha20-poly1305@openssh.com'
set service ssh ciphers 'aes128-ctr'
set service ssh ciphers 'aes128-gcm@openssh.com'
set service ssh ciphers 'aes192-cbc'
set service ssh ciphers 'aes192-ctr'
set service ssh ciphers 'aes256-cbc'
set service ssh ciphers 'aes256-ctr'
set service ssh ciphers 'aes256-gcm@openssh.com'
set service ssh key-exchange 'curve25519-sha256'
set service ssh key-exchange 'curve25519-sha256@libssh.org'
set service ssh key-exchange 'diffie-hellman-group-exchange-sha256'
set service ssh key-exchange 'diffie-hellman-group14-sha256'
set service ssh key-exchange 'diffie-hellman-group16-sha512'
set service ssh key-exchange 'diffie-hellman-group18-sha512'
set service ssh mac 'hmac-sha2-256'
set service ssh mac 'hmac-sha2-256-etm@openssh.com'
set service ssh mac 'hmac-sha2-512'
set service ssh mac 'hmac-sha2-512-etm@openssh.com'

Set timezone & ntp server from global pool:

set system time-zone 'Europe/Amsterdam'
delete system ntp
set service ntp server 0.nl.pool.ntp.org
set service ntp server 1.nl.pool.ntp.org

Enable DNS from DHCP, also for local machine. Set cache to 100k, 10x up from dnsmasq default. More local caching should give more speed and more privay (less public querying).

# N.B. this should be in sync with the WAN interface you have defined above, e.g. pppoe0
# Specifically use name servers received for the interface that is using DHCP client to get an IP
set service dns forwarding allow-from '172.17.0.0/16'
set service dns forwarding cache-size '100004'
set service dns forwarding dhcp 'eth1'
set service dns forwarding listen-address '172.17.10.1'
set service dns forwarding listen-address '172.17.20.1'
set service dns forwarding listen-address '172.17.30.1'
set service dns forwarding listen-address '172.17.40.1'
set service dns forwarding listen-address '172.17.50.1'
# set system name-server 172.17.10.1 # use static as alternative

Configure DHCP server ranges for VLAN. 100-254 is dynamic, 1-100 is for static hosts.

delete service dhcp-server shared-network-name vlan10
set service dhcp-server shared-network-name vlan10 authoritative
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 option default-router '172.17.10.1'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 option domain-name 'lan.vanwerkhoven.org'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 option name-server '172.17.10.1'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 range vlan10range start '172.17.10.100'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 range vlan10range stop '172.17.10.254'
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 subnet-id '1'

delete service dhcp-server shared-network-name vlan20
set service dhcp-server shared-network-name vlan20 authoritative
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 option default-router '172.17.20.1'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 option domain-name 'lan.vanwerkhoven.org'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 option name-server '172.17.20.1'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 range vlan20range start '172.17.20.100'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 range vlan20range stop '172.17.20.254'
set service dhcp-server shared-network-name vlan20 subnet 172.17.20.0/24 subnet-id '2'

delete service dhcp-server shared-network-name vlan30
set service dhcp-server shared-network-name vlan30 authoritative
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 option default-router '172.17.30.1'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 option domain-name 'lan.vanwerkhoven.org'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 option name-server '172.17.30.1'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 range vlan30range start '172.17.30.100'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 range vlan30range stop '172.17.30.254'
set service dhcp-server shared-network-name vlan30 subnet 172.17.30.0/24 subnet-id '3'

set service dhcp-server shared-network-name vlan40 authoritative
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 option default-router '172.17.40.1'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 option domain-name 'lan.vanwerkhoven.org'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 option name-server '172.17.40.1'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 range vlan40range start '172.17.40.100'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 range vlan40range stop '172.17.40.254'
set service dhcp-server shared-network-name vlan40 subnet 172.17.40.0/24 subnet-id '4'

Set up masquerading for outbound traffic. Ensure high rule number so it’s processed after firewalling.

# N.B. this should be in sync with the WAN interface you have defined above, e.g. pppoe0
set nat source rule 5010 description 'Masquerade for WAN'
set nat source rule 5010 outbound-interface name 'pppoe0'
set nat source rule 5010 protocol 'all'
set nat source rule 5010 source address '172.17.0.0/16'
set nat source rule 5010 translation address 'masquerade'

Set up static ips/host names. Still not possible to set the hostname of the router itself (reddit.com).

# Internal facing services
set system static-host-mapping host-name vyos.lan.vanwerkhoven.org inet 172.17.10.1 # this works only in >1.4, previously the device hostname always returned 127.0.0.1
set system static-host-mapping host-name proteus.lan.vanwerkhoven.org inet 172.17.10.2

Split DNS for specific hosts (mostly https & ssh) N.B. Ensure you restart DNS or set dns cache-size to 0 to ensure the cache is cleared! Public & internal facing services requiring https - to use letsencrypt we can only have 1 subdomain below vanwerkhoven.org, e.g. service.lan.vanwerkhoven.org does not work with a letencrypt certificate for *.vanwerkhoven.org.

delete system static-host-mapping host-name ssh.vanwerkhoven.org inet
set system static-host-mapping host-name ssh.vanwerkhoven.org inet 172.17.10.7
delete system static-host-mapping host-name home.vanwerkhoven.org inet
set system static-host-mapping host-name home.vanwerkhoven.org inet 172.17.10.7
delete system static-host-mapping host-name www.vanwerkhoven.org inet
set system static-host-mapping host-name www.vanwerkhoven.org inet 172.17.10.7
delete system static-host-mapping host-name photos.vanwerkhoven.org inet
set system static-host-mapping host-name photos.vanwerkhoven.org inet 172.17.10.7
delete system static-host-mapping host-name grafana.vanwerkhoven.org inet
set system static-host-mapping host-name grafana.vanwerkhoven.org inet 172.17.10.7
delete system static-host-mapping host-name nextcloud.vanwerkhoven.org inet
set system static-host-mapping host-name nextcloud.vanwerkhoven.org inet 172.17.10.7

Set hard coded IP for some hosts

set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping gs108e ip-address 172.17.10.3
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping gs108e mac 78:D2:94:2F:81:F8
set system static-host-mapping host-name gs108e.lan.vanwerkhoven.org inet 172.17.10.3

set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping uap-lr1-floor1 ip-address 172.17.10.10
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping uap-lr1-floor1 mac 18:E8:29:93:E1:66
set system static-host-mapping host-name uap-lr1-floor1.lan.vanwerkhoven.org inet 172.17.10.10

set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping uap-lr2-floor2 ip-address 172.17.10.11
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping uap-lr2-floor2 mac 18:E8:29:E6:00:2E
set system static-host-mapping host-name uap-lr2-floor2.lan.vanwerkhoven.org inet 172.17.10.11

set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping uap-lr3-floor3 ip-address 172.17.10.12
set service dhcp-server shared-network-name vlan10 subnet 172.17.10.0/24 static-mapping uap-lr3-floor3 mac E0:63:DA:79:A7:93
set system static-host-mapping host-name uap-lr3-floor3.lan.vanwerkhoven.org inet 172.17.10.12

Setup IPv6 on KPN PPPoE - TODO ¶

Based on https://blog.daknob.net/ipv6-first-with-vyos/ (daknob.net) and https://docs.vyos.io/en/latest/configexamples/pppoe-ipv6-basic.html (vyos.io)

set interfaces pppoe pppoe0 ipv6 address autoconf # needed sometimes
set interfaces pppoe pppoe0 ipv adjust-mss 'clamp-mss-to-pmtu' # because of PPPoE?

set interfaces pppoe pppoe0 dhcpv6-options pd 1 length '48' # this is needed, else the default value (/64) might conflict with that the ISP gives (/48) which is then interpreted as a /64 address.
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.10 sla-id 16
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.10 address 1

set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.20 sla-id 32
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.20 address 1

set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.30 sla-id 48
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.30 address 1

set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.40 sla-id 64
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface br100.40 address 1

# set service dns forwarding dnssec validate
set service dns forwarding listen-address 2a02:a47f:e000::53
set service dns forwarding listen-address 2a02:a47f:e000::54

set service router-advert interface br100.10 link-mtu '1492'
set service router-advert interface br100.10 other-config-flag
set service router-advert interface br100.10 name-server 2a02:a47f:e000::53
set service router-advert interface br100.10 name-server 2a02:a47f:e000::54
set service router-advert interface br100.10 prefix ::/64 valid-lifetime '172800'

set service router-advert interface br100.20 link-mtu '1492'
set service router-advert interface br100.20 other-config-flag
set service router-advert interface br100.20 name-server 2a02:a47f:e000::53
set service router-advert interface br100.20 name-server 2a02:a47f:e000::54
set service router-advert interface br100.20 prefix ::/64 valid-lifetime '172800'

set service router-advert interface br100.30 link-mtu '1492'
set service router-advert interface br100.30 other-config-flag
set service router-advert interface br100.30 name-server 2a02:a47f:e000::53
set service router-advert interface br100.30 name-server 2a02:a47f:e000::54
set service router-advert interface br100.30 prefix ::/64 valid-lifetime '172800'

set service router-advert interface br100.40 link-mtu '1492'
set service router-advert interface br100.40 other-config-flag
set service router-advert interface br100.40 name-server 2a02:a47f:e000::53
set service router-advert interface br100.40 name-server 2a02:a47f:e000::54
set service router-advert interface br100.40 prefix ::/64 valid-lifetime '172800'

set firewall group interface-group stateful-fw interface br100.10
set firewall group interface-group stateful-fw interface br100.20
set firewall group interface-group stateful-fw interface br100.30
set firewall group interface-group stateful-fw interface br100.40

set firewall ipv6 forward filter rule 10 action accept
set firewall ipv6 forward filter rule 10 outbound-interface group stateful-fw
set firewall ipv6 forward filter rule 10 state established
set firewall ipv6 forward filter rule 10 state related

set firewall ipv6 forward filter rule 20 action accept
set firewall ipv6 forward filter rule 20 outbound-interface group stateful-fw
set firewall ipv6 forward filter rule 20 protocol ipv6-icmp

set firewall ipv6 forward filter rule 99 action drop
set firewall ipv6 forward filter rule 99 outbound-interface group stateful-fw


# Enable DNS https://blog.daknob.net/ipv6-first-with-vyos/
set service dns forwarding dnssec validate
# set service dns forwarding ignore-hosts-file
set service dns forwarding dns64-prefix 64:ff9b::/96 # what is needed?

set service dns forwarding listen-address fd53::53
# set service dns forwarding listen-address 192.168.1.1

# Distribute IPs to clients
set service router-advert interface br100.10 name-server fd53::53
set service router-advert interface br100.10 prefix ::/64 valid-lifetime '172800'
set service router-advert interface br100.10 link-mtu '1492'

# set service router-advert interface br100.10 nat64prefix 64:ff9b::/96

# set service router-advert interface br100.10 link-mtu '1492'
# set service router-advert interface br100.10 name-server <NAME SERVER> -- how can we use ISP?
# set service router-advert interface br100.10 prefix ::/64 valid-lifetime '172800'

QoS & NAT ¶

Update QoS, DNS & firewall, these can also persist and be set at the same time

set qos interface eth1.300 egress 'WAN_QUEUE'
set qos interface pppoe0 egress 'WAN_QUEUE'
set qos interface eth1 egress 'WAN_QUEUE'

set service dns forwarding dhcp 'eth1.300'
set service dns forwarding dhcp 'pppoe0' 
#-- not applicable for pppoe -- why not?
set service dns forwarding dhcp 'eth1'

### Set firewall, these can persist in an interface-group or listed separately

set firewall group interface-group WAN interface eth1.300
set firewall group interface-group WAN interface pppoe0
set firewall group interface-group WAN interface eth1

# set firewall zone WAN interface WAN
set firewall zone WAN interface 'eth1.300'
set firewall zone WAN interface 'pppoe0'
set firewall zone WAN interface 'eth1'

Set NAT, this needs to be adjusted per outbound VLAN

set nat destination rule 100 inbound-interface 'eth1.300'
set nat destination rule 102 inbound-interface 'eth1.300'
set nat destination rule 104 inbound-interface 'eth1.300'
set nat destination rule 106 inbound-interface 'eth1.300'
set nat destination rule 108 inbound-interface 'eth1.300'
set nat source rule 5001 outbound-interface 'eth1.300'
set nat source rule 5010 outbound-interface 'eth1.300'

set nat destination rule 100 inbound-interface 'pppoe0'
set nat destination rule 102 inbound-interface 'pppoe0'
set nat destination rule 104 inbound-interface 'pppoe0'
set nat destination rule 106 inbound-interface 'pppoe0'
set nat destination rule 108 inbound-interface 'pppoe0'
set nat source rule 5001 outbound-interface 'pppoe0'
set nat source rule 5010 outbound-interface 'pppoe0'

set nat destination rule 100 inbound-interface 'eth1'
set nat destination rule 102 inbound-interface 'eth1'
set nat destination rule 104 inbound-interface 'eth1'
set nat destination rule 106 inbound-interface 'eth1'
set nat destination rule 108 inbound-interface 'eth1'
set nat source rule 5001 outbound-interface 'eth1'
set nat source rule 5010 outbound-interface 'eth1'

delete interfaces ethernet eth1 mtu
delete interfaces ethernet eth1 vif 6 mtu
delete interfaces pppoe pppoe0 mtu
delete interfaces ethernet eth1 vif 6 ip adjust-mss

Set up port forwaring ¶

10022 to 172.17.10.4:22
443 to 172.17.10.2:443
80 to 172.17.10.2:80
8883 to 172.17.10.2:1883

set nat destination rule 100 description 'Port Forward: SSH to 172.17.10.7'
set nat destination rule 100 destination port '22'
set nat destination rule 100 inbound-interface name 'pppoe0'
set nat destination rule 100 protocol 'tcp'
set nat destination rule 100 translation address '172.17.10.7'

set nat destination rule 102 description 'Port Forward: HTTP to 172.17.10.7'
set nat destination rule 102 destination port '80'
set nat destination rule 102 inbound-interface name 'pppoe0'
set nat destination rule 102 protocol 'tcp'
set nat destination rule 102 translation address '172.17.10.7'

set nat destination rule 104 description 'Port Forward: HTTPS to 172.17.10.7'
set nat destination rule 104 destination port '443'
set nat destination rule 104 inbound-interface name 'pppoe0'
set nat destination rule 104 protocol 'tcp'
set nat destination rule 104 translation address '172.17.10.7'

set nat destination rule 106 description 'Port Forward: MQTT to 172.17.10.7'
set nat destination rule 106 destination port '1883'
set nat destination rule 106 inbound-interface name 'pppoe0'
set nat destination rule 106 protocol 'tcp'
set nat destination rule 106 translation address '172.17.10.7'
set nat destination rule 106 translation port '1883'

set nat destination rule 108 description 'Port Forward: MQTTS to 172.17.10.7'
set nat destination rule 108 destination port '8883'
set nat destination rule 108 inbound-interface name 'pppoe0'
set nat destination rule 108 protocol 'tcp'
set nat destination rule 108 translation address '172.17.10.7'

Hairpin NAT (vyos.io) is not implemented (vyos.net) well for dynamics IPs (see here (reddit.com) and here (vyos.io)), so we use split DNS for local resolving instead.

For reference, a hairpin NAT setup looks like:

set nat destination rule 100 description 'Port Forward SSH'
set nat destination rule 100 destination port '22'
set nat destination rule 100 inbound-interface '<WAN interface>' # WAN interface
set nat destination rule 100 protocol 'tcp'
set nat destination rule 100 translation address '<LAN IP>' # LAN IP

set nat destination rule 101 description 'Port Forward: SSH (NAT Reflection: INSIDE)'
set nat destination rule 101 destination port '22'
set nat destination rule 101 destination address '<WAN IP>' # WAN IP --> required but not in official docs
set nat destination rule 101 inbound-interface '<LAN interface>' # LAN interface
set nat destination rule 101 protocol 'tcp'
set nat destination rule 101 translation address '<LAN IP>' # LAN IP

set nat source rule 100 description 'Port Forward: all to <LAN RANGE>/24 (NAT Reflection: INSIDE)'
set nat source rule 100 destination address '<LAN RANGE>/24'
set nat source rule 100 source address '<LAN RANGE>/24'
set nat source rule 100 outbound-interface '<LAN interface>' # LAN interface
set nat source rule 100 protocol 'tcp'
set nat source rule 100 translation address 'masquerade'

Set up zone-based firewall ¶

Set up zone-based firewall using the following zones:

WAN: Internet
Local: router itself, access to everything (VPN, DNS, DHCP, etc.)
Infra: trusted VLAN with infrastructure, access to everything (switch, server, pve, access points)
Trusted: trusted clients, limited access to infra (e.g. home laptops, appletv, phones, ipad, hue)
Guest: untrusted clients, only access to WAN (e.g. work laptops, work phones, guest phones, thermostat)
IoT: untrusted clients, only access to server in Infra, no WAN access (esp clients)

Rules:

FW_ACCEPT: drop invalid, accept rest
FW_DROP: drop all
FW_2LOCAL: allow DNS, DHCP, SSH, Wireguard (to router)
FW_TRUST2INFRA: allow trusted clients to reach: server (SSH, HTTP, HTTPS, Home Assistant (via proxy?), Grafana (via proxy?)), pve (SSH), unifi (web only?)
FW_IOT2INFRA: allow IOT to reach server (MQTT(S)/Home Assistant API/HTTP(S))
FW_GUEST2TRUST: allow guest clients to reach: appleTV (all ports, mdns)
FW_WAN2ALL: allow established & related, drop rest
FW_WAN2INFRA: allow established & related, allow port forwards, drop rest. For port forwards we only have to specify the port, as the IP is implied by the port forwarding rule set up earlier (e.g. allowing port 80 doesn’t open http on all hosts because the forward only allows to 1 specific host)
FW_WAN2INFRA: allow established & related, allow wireguard (maybe IKEv2 later), drop rest.

I use the following policy table from one zone to another:

	to Local	to Infra	to Trusted	to Guest	to IoT	to WAN
Local		FW_ACCEPT	FW_ACCEPT	FW_ACCEPT	FW_ACCEPT	FW_ACCEPT
Infra	FW_ACCEPT		FW_ACCEPT	FW_ACCEPT	FW_ACCEPT	FW_ACCEPT
Trust.	FW_2LOCAL	FW_TRUST2INFRA		FW_ACCEPT	FW_ACCEPT	FW_ACCEPT
Guest	FW_2LOCAL	FW_GUEST2INFRA	FW_GUEST2TRUST		FW_DROP	FW_ACCEPT
IoT	FW_2LOCAL	FW_IOT2INFRA	FW_IOT2INFRA	FW_DROP		FW_IOT2WAN
WAN	FW_WAN2LOCAL	FW_WAN2INFRA	FW_WAN2ALL	FW_WAN2ALL	FW_WAN2ALL

FW_ACCEPT

set firewall ipv4 name FW_ACCEPT default-action 'return'

set firewall ipv4 name FW_ACCEPT rule 200 action 'drop'
set firewall ipv4 name FW_ACCEPT rule 200 description 'drop invalid'
set firewall ipv4 name FW_ACCEPT rule 200 state 'invalid'

FW_DROP

set firewall ipv4 name FW_DROP default-action 'drop'

FW_WAN2ALL

set firewall ipv4 name FW_WAN2ALL default-action 'drop'
set firewall ipv4 name FW_WAN2ALL rule 200 action 'return'
set firewall ipv4 name FW_WAN2ALL rule 200 description 'accept established/related'
set firewall ipv4 name FW_WAN2ALL rule 200 state 'established'
set firewall ipv4 name FW_WAN2ALL rule 200 state 'related'

FW_WAN2LOCAL

set firewall ipv4 name FW_WAN2LOCAL default-action 'drop'
set firewall ipv4 name FW_WAN2LOCAL default-log
set firewall ipv4 name FW_WAN2LOCAL rule 200 action 'return'
set firewall ipv4 name FW_WAN2LOCAL rule 200 description 'accept established/related'
set firewall ipv4 name FW_WAN2LOCAL rule 200 log
set firewall ipv4 name FW_WAN2LOCAL rule 200 state 'established'
set firewall ipv4 name FW_WAN2LOCAL rule 200 state 'related'

set firewall ipv4 name FW_WAN2LOCAL rule 210 action 'return'
set firewall ipv4 name FW_WAN2LOCAL rule 210 description 'wireguard'
set firewall ipv4 name FW_WAN2LOCAL rule 210 destination port '51820'
set firewall ipv4 name FW_WAN2LOCAL rule 210 log
set firewall ipv4 name FW_WAN2LOCAL rule 210 protocol 'udp'
set firewall ipv4 name FW_WAN2LOCAL rule 210 state 'new'

FW_WAN2INFRA

set firewall ipv4 name FW_WAN2INFRA default-action 'drop'

set firewall ipv4 name FW_WAN2INFRA rule 200 action 'return'
set firewall ipv4 name FW_WAN2INFRA rule 200 description 'accept established/related'
set firewall ipv4 name FW_WAN2INFRA rule 200 state 'established'
set firewall ipv4 name FW_WAN2INFRA rule 200 state 'related'

set firewall ipv4 name FW_WAN2INFRA rule 210 action 'return'
set firewall ipv4 name FW_WAN2INFRA rule 210 description 'accept port forwards'
set firewall ipv4 name FW_WAN2INFRA rule 210 destination port '22,80,443,1883'
set firewall ipv4 name FW_WAN2INFRA rule 210 log
set firewall ipv4 name FW_WAN2INFRA rule 210 protocol 'tcp'
set firewall ipv4 name FW_WAN2INFRA rule 210 state 'new'

FW_2LOCAL

set firewall ipv4 name FW_2LOCAL default-action 'drop'

set firewall ipv4 name FW_2LOCAL rule 200 action 'return'
set firewall ipv4 name FW_2LOCAL rule 200 description 'accept established/related'
set firewall ipv4 name FW_2LOCAL rule 200 state 'established'
set firewall ipv4 name FW_2LOCAL rule 200 state 'related'

set firewall ipv4 name FW_2LOCAL rule 210 action 'return'
set firewall ipv4 name FW_2LOCAL rule 210 description 'accept dhcp'
set firewall ipv4 name FW_2LOCAL rule 210 destination port '67-68'
set firewall ipv4 name FW_2LOCAL rule 210 protocol 'udp'

set firewall ipv4 name FW_2LOCAL rule 220 action 'return'
set firewall ipv4 name FW_2LOCAL rule 220 description 'accept dns'
set firewall ipv4 name FW_2LOCAL rule 220 destination port '53'
set firewall ipv4 name FW_2LOCAL rule 220 protocol 'udp'

set firewall ipv4 name FW_2LOCAL rule 230 action 'return'
set firewall ipv4 name FW_2LOCAL rule 230 description 'accept ssh'
set firewall ipv4 name FW_2LOCAL rule 230 destination port '22'
set firewall ipv4 name FW_2LOCAL rule 230 protocol 'tcp'

FW_TRUST2INFRA

set firewall ipv4 name FW_TRUST2INFRA default-action 'drop'

set firewall ipv4 name FW_TRUST2INFRA rule 200 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 200 description 'accept established/related'
set firewall ipv4 name FW_TRUST2INFRA rule 200 state 'established'
set firewall ipv4 name FW_TRUST2INFRA rule 200 state 'related'

set firewall ipv4 name FW_TRUST2INFRA rule 210 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 210 description 'accept mqtt(s)/http(s)/HA/ssh/grafana/jellyfin&emby/plex/iperf/transmission to proteus'
set firewall ipv4 name FW_TRUST2INFRA rule 210 destination address '172.17.10.2'
set firewall ipv4 name FW_TRUST2INFRA rule 210 destination port '8883,1883,80,443,8123,22,3000,8096,32400,32469,7575,9091'
set firewall ipv4 name FW_TRUST2INFRA rule 210 protocol 'tcp'

set firewall ipv4 name FW_TRUST2INFRA rule 211 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 211 description 'accept GDM network discovery to proteus'
set firewall ipv4 name FW_TRUST2INFRA rule 211 destination address '172.17.10.2'
set firewall ipv4 name FW_TRUST2INFRA rule 211 destination port '1900,1901,5353,32410,32412,32413,32414'
set firewall ipv4 name FW_TRUST2INFRA rule 211 protocol 'udp'

set firewall ipv4 name FW_TRUST2INFRA rule 212 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 212 description 'accept ssh,http(s) to proteus2'
set firewall ipv4 name FW_TRUST2INFRA rule 212 destination address '172.17.10.7'
set firewall ipv4 name FW_TRUST2INFRA rule 212 destination port '22,80,443'
set firewall ipv4 name FW_TRUST2INFRA rule 212 protocol 'tcp'

set firewall ipv4 name FW_TRUST2INFRA rule 220 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 220 description 'accept ssh to pve'
set firewall ipv4 name FW_TRUST2INFRA rule 220 destination address '172.17.10.4'
set firewall ipv4 name FW_TRUST2INFRA rule 220 destination port '22'
set firewall ipv4 name FW_TRUST2INFRA rule 220 protocol 'tcp'

set firewall ipv4 name FW_TRUST2INFRA rule 230 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 230 description 'accept ssh to unifi controller'
set firewall ipv4 name FW_TRUST2INFRA rule 230 destination address '172.17.10.5'
set firewall ipv4 name FW_TRUST2INFRA rule 230 destination port '22,443,8443'
set firewall ipv4 name FW_TRUST2INFRA rule 230 protocol 'tcp'

set firewall ipv4 name FW_TRUST2INFRA rule 240 action 'return'
set firewall ipv4 name FW_TRUST2INFRA rule 240 description 'accept HA to home assistant'
set firewall ipv4 name FW_TRUST2INFRA rule 240 destination address '172.17.10.20'
set firewall ipv4 name FW_TRUST2INFRA rule 240 destination port '8123'
set firewall ipv4 name FW_TRUST2INFRA rule 240 protocol 'tcp'

set firewall ipv4 name FW_TRUST2INFRA rule 250 action 'accept'
set firewall ipv4 name FW_TRUST2INFRA rule 250 description 'accept mqtt(s)/http(s)/HA/ssh/grafana/jellyfin&emby/iperf/transmission to proteus2'
set firewall ipv4 name FW_TRUST2INFRA rule 250 destination address '172.17.10.7'
set firewall ipv4 name FW_TRUST2INFRA rule 250 destination port '8883,1883,80,443,8123,22,3000,8096,7575,9091'
set firewall ipv4 name FW_TRUST2INFRA rule 250 protocol 'tcp'

set firewall ipv4 name FW_TRUST2INFRA rule 251 action 'accept'
set firewall ipv4 name FW_TRUST2INFRA rule 251 description 'accept nextcloud/pigallery2 debug ports to proteus2'
set firewall ipv4 name FW_TRUST2INFRA rule 251 destination address '172.17.10.7'
set firewall ipv4 name FW_TRUST2INFRA rule 251 destination port '9081,3080'
set firewall ipv4 name FW_TRUST2INFRA rule 251 protocol 'tcp'

FW_GUEST2INFRA

set firewall ipv4 name FW_GUEST2INFRA default-action 'drop'

set firewall ipv4 name FW_GUEST2INFRA rule 200 action 'return'
set firewall ipv4 name FW_GUEST2INFRA rule 200 description 'accept established/related'
set firewall ipv4 name FW_GUEST2INFRA rule 200 state 'established'
set firewall ipv4 name FW_GUEST2INFRA rule 200 state 'related'

set firewall ipv4 name FW_GUEST2INFRA rule 210 action 'return'
set firewall ipv4 name FW_GUEST2INFRA rule 210 description 'accept http(s)/ssh to proteus'
set firewall ipv4 name FW_GUEST2INFRA rule 210 destination address '172.17.10.2'
set firewall ipv4 name FW_GUEST2INFRA rule 210 destination port '22,80,443'
set firewall ipv4 name FW_GUEST2INFRA rule 210 protocol 'tcp'

set firewall ipv4 name FW_GUEST2INFRA rule 220 action 'return'
set firewall ipv4 name FW_GUEST2INFRA rule 220 description 'accept http(s)/ssh to pve'
set firewall ipv4 name FW_GUEST2INFRA rule 220 destination address '172.17.10.4'
set firewall ipv4 name FW_GUEST2INFRA rule 220 destination port '22,80,443'
set firewall ipv4 name FW_GUEST2INFRA rule 220 protocol 'tcp'

FW_IOT2INFRA

set firewall ipv4 name FW_IOT2INFRA default-action 'drop'

set firewall ipv4 name FW_IOT2INFRA rule 200 action 'return'
set firewall ipv4 name FW_IOT2INFRA rule 200 description 'accept established/related'
set firewall ipv4 name FW_IOT2INFRA rule 200 state 'established'
set firewall ipv4 name FW_IOT2INFRA rule 200 state 'related'

set firewall ipv4 name FW_IOT2INFRA rule 210 action 'return'
set firewall ipv4 name FW_IOT2INFRA rule 210 description 'accept mqtt(s)/HA API to proteus2'
set firewall ipv4 name FW_IOT2INFRA rule 210 destination address '172.17.10.7'
set firewall ipv4 name FW_IOT2INFRA rule 210 destination port '8883,1883,6053'
set firewall ipv4 name FW_IOT2INFRA rule 210 protocol 'tcp'

# not needed anymore?
# set firewall ipv4 name FW_IOT2INFRA rule 220 action 'return'
# set firewall ipv4 name FW_IOT2INFRA rule 220 description 'accept NibeGW UDP traffic to proteus'
# set firewall ipv4 name FW_IOT2INFRA rule 220 destination address '172.17.10.2'
# set firewall ipv4 name FW_IOT2INFRA rule 220 destination port '9999,10000'
# set firewall ipv4 name FW_IOT2INFRA rule 220 protocol 'udp'

FW_IOT2WAN

set firewall ipv4 name FW_IOT2WAN default-action 'drop'

set firewall ipv4 name FW_IOT2WAN rule 200 action 'return'
set firewall ipv4 name FW_IOT2WAN rule 200 description 'accept established/related'
set firewall ipv4 name FW_IOT2WAN rule 200 state 'established'
set firewall ipv4 name FW_IOT2WAN rule 200 state 'related'

set firewall ipv4 name FW_IOT2WAN rule 210 action 'return'
set firewall ipv4 name FW_IOT2WAN rule 210 description 'accept solaredge to phone home'
set firewall ipv4 name FW_IOT2WAN rule 210 protocol 'tcp'
set firewall ipv4 name FW_IOT2WAN rule 210 source address '172.17.40.35'

FW_GUEST2TRUST

set firewall ipv4 name FW_GUEST2TRUST default-action 'drop'

set firewall ipv4 name FW_GUEST2TRUST rule 200 action 'return'
set firewall ipv4 name FW_GUEST2TRUST rule 200 description 'accept established/related'
set firewall ipv4 name FW_GUEST2TRUST rule 200 state 'established'
set firewall ipv4 name FW_GUEST2TRUST rule 200 state 'related'

set firewall ipv4 name FW_GUEST2TRUST rule 210 action 'return'
set firewall ipv4 name FW_GUEST2TRUST rule 210 description 'accept access to AppleTV'
set firewall ipv4 name FW_GUEST2TRUST rule 210 destination address '172.17.20.20-172.17.20.21'
set firewall ipv4 name FW_GUEST2TRUST rule 210 protocol 'tcp_udp'

Apply firewall zones to interfaces

	to Local	to Infra	to Trusted	to Guest	to IoT	to WAN
Local		FW_ACCEPT	FW_ACCEPT	FW_ACCEPT	FW_ACCEPT	FW_ACCEPT
Infra	FW_ACCEPT		FW_ACCEPT	FW_ACCEPT	FW_ACCEPT	FW_ACCEPT
Trust.	FW_2LOCAL	FW_TRUST2INFRA		FW_ACCEPT	FW_ACCEPT	FW_ACCEPT
Guest	FW_2LOCAL	FW_GUEST2INFRA	FW_GUEST2TRUST		FW_DROP	FW_ACCEPT
IoT	FW_2LOCAL	FW_IOT2INFRA	FW_IOT2INFRA	FW_DROP		FW_IOT2WAN
WAN	FW_WAN2LOCAL	FW_WAN2INFRA	FW_WAN2ALL	FW_WAN2ALL	FW_WAN2ALL

N.B. this should be in sync with the WAN interface you have defined above, e.g. pppoe0

# Temp disable firewall
delete firewall zone LOCAL
delete firewall zone INFRA
delete firewall zone TRUSTED
delete firewall zone GUEST
delete firewall zone IOT
delete firewall zone WAN

set firewall zone LOCAL default-action 'drop'
set firewall zone LOCAL from GUEST firewall name 'FW_2LOCAL'
set firewall zone LOCAL from INFRA firewall name 'FW_ACCEPT'
set firewall zone LOCAL from IOT firewall name 'FW_2LOCAL'
set firewall zone LOCAL from TRUSTED firewall name 'FW_2LOCAL'
set firewall zone LOCAL from WAN firewall name 'FW_WAN2LOCAL'
set firewall zone LOCAL local-zone

set firewall zone INFRA default-action 'drop'
set firewall zone INFRA from GUEST firewall name 'FW_GUEST2INFRA'
set firewall zone INFRA from IOT firewall name 'FW_IOT2INFRA'
set firewall zone INFRA from LOCAL firewall name 'FW_ACCEPT'
set firewall zone INFRA from TRUSTED firewall name 'FW_TRUST2INFRA'
set firewall zone INFRA from WAN firewall name 'FW_WAN2INFRA'
set firewall zone INFRA interface 'br100.10'

set firewall zone TRUSTED default-action 'drop'
set firewall zone TRUSTED from GUEST firewall name 'FW_GUEST2TRUST'
set firewall zone TRUSTED from INFRA firewall name 'FW_ACCEPT'
set firewall zone TRUSTED from IOT firewall name 'FW_IOT2INFRA'
set firewall zone TRUSTED from LOCAL firewall name 'FW_ACCEPT'
set firewall zone TRUSTED from WAN firewall name 'FW_WAN2ALL'
set firewall zone TRUSTED interface 'br100.20'
set firewall zone TRUSTED interface 'wg0'

set firewall zone GUEST default-action 'drop'
set firewall zone GUEST from INFRA firewall name 'FW_ACCEPT'
set firewall zone GUEST from IOT firewall name 'FW_DROP'
set firewall zone GUEST from LOCAL firewall name 'FW_ACCEPT'
set firewall zone GUEST from TRUSTED firewall name 'FW_ACCEPT'
set firewall zone GUEST from WAN firewall name 'FW_WAN2ALL'
set firewall zone GUEST interface 'br100.30'

set firewall zone IOT default-action 'drop'
set firewall zone IOT from GUEST firewall name 'FW_DROP'
set firewall zone IOT from INFRA firewall name 'FW_ACCEPT'
set firewall zone IOT from LOCAL firewall name 'FW_ACCEPT'
set firewall zone IOT from TRUSTED firewall name 'FW_ACCEPT'
set firewall zone IOT from WAN firewall name 'FW_WAN2ALL'
set firewall zone IOT interface 'br100.40'

set firewall zone WAN default-action 'drop'
set firewall zone WAN from GUEST firewall name 'FW_ACCEPT'
set firewall zone WAN from INFRA firewall name 'FW_ACCEPT'
set firewall zone WAN from IOT firewall name 'FW_IOT2WAN'
set firewall zone WAN from LOCAL firewall name 'FW_ACCEPT'
set firewall zone WAN from TRUSTED firewall name 'FW_ACCEPT'
set firewall zone WAN interface 'eth1'
set firewall zone WAN interface 'pppoe0'

Set generic firewall settings (these might be defaults):

set firewall global-options all-ping 'enable'
set firewall global-options broadcast-ping 'disable'
set firewall global-options ip-src-route 'disable'
set firewall global-options ipv6-receive-redirects 'disable'
set firewall global-options ipv6-src-route 'disable'
set firewall global-options log-martians 'enable'
set firewall global-options receive-redirects 'disable'
set firewall global-options send-redirects 'enable'
set firewall global-options source-validation 'disable'
set firewall global-options syn-cookies 'enable'

Restart firewall - is done automatically. If you don’t notice the changes you probably made a mistake :p

Allow mDNS reflector, allow guests to reach AppleTV (.30 to .20) & Plex to reach AppleTV (.10 to .20). Also set igmp-proxy for Plex media server (maybe - see https://www.reddit.com/r/Ubiquiti/comments/6rdqx8/plex_server_across_multiple_subnets/) (reddit.com). Not sure if this is working.

set service mdns repeater interface br100.10
set service mdns repeater interface br100.20
set service mdns repeater interface br100.30

# Not working yet
# set protocols igmp-proxy interface br100.10 role upstream
# set protocols igmp-proxy interface br100.10 alt-subnet 172.17.0.0/16
# set protocols igmp-proxy interface br100.20 role downstream
# set protocols igmp-proxy interface br100.30 role downstream

# Not working yet
# set service broadcast-relay id 1 description 'Plex GDM'
# set service broadcast-relay id 1 interface 'br100.10'
# set service broadcast-relay id 1 interface 'br100.20'
# set service broadcast-relay id 1 port '32410'
# set service broadcast-relay id 2 description 'Plex GDM'
# set service broadcast-relay id 2 interface 'br100.10'
# set service broadcast-relay id 2 interface 'br100.20'
# set service broadcast-relay id 2 port '32412'
# set service broadcast-relay id 3 description 'Plex GDM'
# set service broadcast-relay id 3 interface 'br100.10'
# set service broadcast-relay id 3 interface 'br100.20'
# set service broadcast-relay id 3 port '32413'
# set service broadcast-relay id 4 description 'Plex GDM'
# set service broadcast-relay id 4 interface 'br100.10'
# set service broadcast-relay id 4 interface 'br100.20'
# set service broadcast-relay id 4 port '32414'
# set service broadcast-relay id 5 description 'Plex GDM'
# set service broadcast-relay id 5 interface 'br100.10'
# set service broadcast-relay id 5 interface 'br100.20'
# set service broadcast-relay id 5 port '32400'

Set up port QoS & MSS ¶

Set up QoS (vyos.io). Partially inspired by this very outdated (2015) source (github.com).


set qos interface pppoe0 egress 'WAN_QUEUE'

set qos policy shaper WAN_QUEUE bandwidth '120mbit'

set qos policy shaper WAN_QUEUE class 10 bandwidth '10%'
set qos policy shaper WAN_QUEUE class 10 match dns ip source port '53'
set qos policy shaper WAN_QUEUE class 10 match icmp ip protocol 'icmp'
set qos policy shaper WAN_QUEUE class 10 priority '1'
set qos policy shaper WAN_QUEUE class 10 queue-type 'fq-codel'

set qos policy shaper WAN_QUEUE default bandwidth '95%'
set qos policy shaper WAN_QUEUE default queue-type 'fq-codel'

Configure Ad-blocking ¶

Set up DNS-based ad blocking on VyOS (vanwerkhoven.org).

Configure VPN ¶

Wireguard ¶

Set up Wireguard VPN, see the official VyOS docs (vyos.io) and this example (reddit.com). We use 172.17.50.0/24 to differentiate from LAN subnets, but keep it on the Trusted VLAN (20).

set firewall name WAN-LOCAL rule 200 state new enable

config
run generate pki wireguard key-pair install interface wg0
# Public key: uGc4JMJ4IJc0aoIY/ITOrFGWjmn+RxnqRQMecOS4uB8=

set interfaces wireguard wg0 address '172.17.50.1/24'
set interfaces wireguard wg0 description "Roadwarrior Wireguard"
set interfaces wireguard wg0 port 51820
set interfaces wireguard wg0 private-key 'SET TO PRIVATE KEY'

# Add first peer with local IP '172.17.50.100/32'
# run generate pki wireguard key-pair
set interfaces wireguard wg0 peer tim public-key 'ZtYsIGNwdEqxB3YY9iUFCnQAOK1IkAsmBhuaZ5cekGI='
# set interfaces wireguard wg0 peer tim persistent-keepalive 15 --> testing if this can be disabled for reduced battery use
set interfaces wireguard wg0 peer tim allowed-ips 172.17.50.100/32

# run generate pki wireguard preshared-key install interface wg0 peer tim
set interfaces wireguard wg0 peer tim preshared-key 'CHANGE TO SECRET PSK'

# Second peer
# run generate pki wireguard key-pair
set interfaces wireguard wg0 peer helene public-key 'XjjkRchCR4qY7i12sJVK/Jl6cycgCovY64305PMw3n8='
# set interfaces wireguard wg0 peer helene persistent-keepalive 15 --> testing if this can be disabled for reduced battery use
set interfaces wireguard wg0 peer helene allowed-ips 172.17.50.101/32

# run generate pki wireguard preshared-key install interface wg0 peer helene
set interfaces wireguard wg0 peer helene preshared-key 'CHANGE TO SECRET PSK'

# Third peer
# run generate pki wireguard key-pair
set interfaces wireguard wg0 peer macbook public-key 'lEKOXMTHRVRURB5aOBioFiZlZS582YYZknqqp/wXt2s='
# set interfaces wireguard wg0 peer macbook persistent-keepalive 15 --> testing if this can be disabled for reduced battery use
set interfaces wireguard wg0 peer macbook allowed-ips 172.17.50.102/32

# run generate pki wireguard preshared-key install interface wg0 peer macbook
set interfaces wireguard wg0 peer macbook preshared-key 'CHANGE TO SECRET PSK'

# Optionally set routing -- only needed for site-to-site LAN setups I think
# set protocols static route 172.17.50.0/24 interface wg0
commit; save; exit

Update Dec 2024: Testing if persistent-keepalive can be disabled. Not sure what the default is if not set explicitly.

Update firewall to not NAT roadwarrior traffic

set service dns forwarding listen-address 172.17.50.1

set nat source rule 5001 description 'Exclude roadwarrior VPN'
set nat source rule 5001 destination address '172.17.50.0/24'
set nat source rule 5001 exclude
set nat source rule 5001 outbound-interface name 'pppoe0'
set nat source rule 5001 protocol 'all'
set nat source rule 5001 translation address 'masquerade'

Now generate client configs and install these on your clients. Be sure to manually copy over the client public key to the server!

generate wireguard client-config tim interface wg0 server home.vanwerkhoven.org address 172.17.50.100/24
generate wireguard client-config helene interface wg0 server home.vanwerkhoven.org address 172.17.50.101/24
generate wireguard client-config macbook interface wg0 server home.vanwerkhoven.org address 172.17.50.102/24

Debug VyOS Wireguard ¶

Enable debug log for Wireguard (vyos.io):

echo module wireguard +p > /sys/kernel/debug/dynamic_debug/control

Check firewall

sudo nft list tables
sudo nft list table vyos_nat
show log firewall name FW_WAN2INFRA
show firewall name FW_WAN2LOCAL

Monitor

monitor traffic interface wg0

IKEv2 IPsec VPN ¶

https://docs.vyos.io/en/latest/configuration/vpn/remoteaccess_ipsec.html (vyos.io)

IKE proposal dh-group / encryption / hash / prf

ESP pfs enable proposal encryption / hash /

Optional: WAN load balancing ¶

e.g for 4G/LTE backup internet or provider change

Some sources:

Possible solution: https://forum.vyos.io/t/wan-failover-with-dhcp/13920 (vyos.io)
Docs: https://docs.vyos.io/en/latest/configuration/loadbalancing/wan.html (vyos.io)
Solution without firewall https://forum.vyos.io/t/pbr-wan-failover/8364/7 (vyos.io)
https://forum.vyos.io/t/wan-load-balancing-failover-conntrack-issues/14266/4 (vyos.io)
Something like this, but not as pseudo-config: https://forum.vyos.io/t/wan-load-balancing-source-destination-group-support/12172 (vyos.io)
https://forum.vyos.io/t/wan-failover-not-working-for-me-rolling-1-4-versions-any-help-please/8886 (vyos.io)
https://forum.vyos.io/t/specifying-outbound-interface-on-destination-translation-nat/8192 (vyos.io)

Extra: PPPoE / MTU debugging ¶

Guides to get KPN PPPoE working

Non working https://community.kpn.com/thuisnetwerk-72/gebruik-een-eigen-router-i-p-v-de-experia-box-458609/index179.html (kpn.com) & https://pastebin.com/S2tWY98e (pastebin.com)
Non working https://gathering.tweakers.net/forum/list_messages/2205126 (tweakers.net)
Working Vyatta https://gist.github.com/Ruben-E/abb9a4a872a7c4ffff058ae291ef2627 (github.com)
Working Vyatta https://jelleraaijmakers.nl/2021/11/kpn-internet-and-iptv-with-edgerouter-x (jelleraaijmakers.nl)
Why you need MSS clamping https://samuel.kadolph.com/2015/02/mtu-and-tcp-mss-when-using-pppoe-2/ (kadolph.com)

# Optional, already arranged via pppoe0
# set interfaces ethernet eth0 vif 6 mtu 1508
set interfaces pppoe pppoe0 description 'KPN WAN'
set interfaces pppoe pppoe0 mtu 1500
set interfaces pppoe pppoe0 authentication username 'internet'
set interfaces pppoe pppoe0 authentication password 'internet'
set interfaces pppoe pppoe0 source-interface 'eth1.6'
# Ensure we get DNS from ISP, i.e. no-peer-dns must be unset
delete interfaces pppoe pppoe0 no-peer-dns
set interfaces ethernet eth1 vif 6 ip adjust-mss clamp-mss-to-pmtu
set interfaces pppoe pppoe0 ip adjust-mss 1352
# set interfaces pppoe pppoe0 ip adjust-mss clamp-mss-to-pmtu

Try clamp MSS: https://docs.vyos.io/en/latest/configuration/interfaces/ethernet.html#cfgcmd-set-interfaces-ethernet-interface-ip-adjust-mss-mss-clamp-mss-to-pmtu (vyos.io) MSS value = MTU - 20 (IP header) - 20 (TCP header), resulting in 1452 bytes on a 1492 byte MTU.

Set MSS-clamping (vyos.io) to ensure optimal link utilization. Diagnose max MTU using ping, then set MSS value = MTU - 20 (IP header) - 20 (TCP header):

set firewall options interface adjust-mss 1460

#Adblock #Debian #Networking #Proxmox #Security #Vyos