My 1 2 3 Offsite Backup Routine

24 Feb, 2025

While I prefer to self-host, I also like to not lose my data. To this end, I backup my stuff on my server, a local hard disk, and a hard disk at an offsite location. I mark parts of the files read-only to have some mild protection against ransomware, and use par2 files to have some bitrot protection for critical datasets.

Backup strategy ¶

Data sources ¶

I have three data sources:

Time Machine MBP
Time Machine MBA
Pictures & projects
Virtual machine images

Data storage ¶

I store backups on three locations:

Server storage (NAS) virtual machine
3TB HDD at home
2x 5TB HDD (rotated home/work)

The first allows me to do frequent backups, e.g. Time Machine, and additionally sync data irregularly that’s always available. The second storage is irregularly updated from the main NAS backup, and rotated with the offsite backup.

Schematically, source/storage is as follows:

	3TB	WD5TB	SG5TB	proteus
TM MBP	N/A	Semi-auto	Semi-auto	Auto
TM MBA	N/A	Manual	Manual	Auto
Photos/projects	Manual	Manual	Manual	Cronjob?
VM images	Manual	Manual	Manual	N/A

Backup protection ¶

I mark critical data as read-only to have some protection against ransomware or accidental deletion. On MacOS I use chflags schg and on the NAS I use chattr +i on the hypervisor (proxmox). Both require a root password, and the NAS files are locked on the hypervisor, which means even root on the virtual machine cannot delete them. I only lock files such that I can still add files to the directories.

Backup scripts ¶

On pve ¶

sudo tar cJvf "/var/lib/vz/dump/pve-conf-$(date +\%Y_\%m_\%d-\%H_\%M_\%S).tar.xz" /etc

Combined ¶

To WD5TB (pics, work, images, TM):

caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/Pictures /Volumes/WD5TB-Data-Tim/; caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs /Volumes/WD5TB-Data-Tim/; caffeinate -im rsync -avH --partial --progress --delete-after pve:/var/lib/vz/dump/ /Volumes/WD5TB-Data-Tim/Images/; caffeinate -im rsync --rsync-path 'sudo -u backupmba rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' proteus2:/mnt/backup/mba/ /Volumes/WD5TB-Data/Helene;

To SG5TB (pics, work, images, TM):

caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/Pictures /Volumes/SG5TB-Data/Tim/; caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs /Volumes/SG5TB-Data/Tim/; caffeinate -im rsync -avH --partial --progress --delete-after pve:/var/lib/vz/dump/ /Volumes/SG5TB-Data/Tim/Images/; caffeinate -im rsync --rsync-path 'sudo -u backupmba rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' proteus2:/mnt/backup/mba/ /Volumes/SG5TB-Data/Helene;

To 3TB (pics, workdocs, images):

caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/Pictures /Volumes/3TB/Tim/; caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs /Volumes/3TB-Data-Enc/; caffeinate -im rsync -avH --partial --progress --delete-after pve:/var/lib/vz/dump/ /Volumes/3TB-Data-Enc/Images/; caffeinate -im rsync --rsync-path 'sudo -u backupmba rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' proteus2:/mnt/backup/mba/ /Volumes/3TB/Helene;

To server (pics, workdocs)

caffeinate -im rsync --rsync-path 'sudo -u backupmbp rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/Pictures proteus2:/mnt/backup/data-tim/; caffeinate -im rsync --rsync-path 'sudo -u backupmbp rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs proteus2:/mnt/backup/data-tim/

From laptop to server ¶

Sync pictures & workdocs to proteus

caffeinate -im rsync --rsync-path 'sudo -u backupmbp rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/Pictures proteus2:/mnt/backup/data-tim/; caffeinate -im rsync --rsync-path 'sudo -u backupmbp rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs proteus2:/mnt/backup/data-tim/

From laptop to disk ¶

caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/Pictures /Volumes/3TB/Tim/; caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs /Volumes/WD5TB-Data-Tim/; caffeinate -im rsync -avH --partial --progress --delete pve:/var/lib/vz/dump/ /Volumes/WD5TB-Data-Tim/Images/; 

caffeinate -im rsync -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' ~/workdocs /Volumes/3TB/Tim/

From server to disk ¶

Sync virtual machine images to disk

caffeinate -im rsync -avH --partial --progress --delete pve:/var/lib/vz/dump/ /Volumes/SG5TB-Data/Tim/Images/;
caffeinate -im rsync -avH --partial --progress --delete pve:/var/lib/vz/dump/ /Volumes/3TB-Data-Enc/Images/;

Sync TM

caffeinate -im rsync --rsync-path 'sudo -u backupmba rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' proteus2:/mnt/backup/mba/ /Volumes/SG5TB-TM-Helene/

caffeinate -im rsync --rsync-path 'sudo -u backupmba rsync' -avH --partial --progress --exclude '.Spotlight-V100' --exclude '.DS_Store*' --exclude '.Trashes' --exclude '.fseventsd' /Volumes/SG5TB-TM-Helene/ proteus2:/mnt/backup/mba/

Lock archive ¶

Prevent accidental deletion / mild ransomware modification prevention.

Lock MacOS disks ¶

On Mac:

find . -type f -print0 | xargs -0 sudo chflags schg
find ~/Pictures/2025 -type f -print0 | xargs -0 sudo chflags schg
find /Volumes/WD5TB-Data/Helene/ -type f -print0 | xargs -0 sudo chflags schg
find /Volumes/WD5TB-Data/Tim/workdocs/ /Volumes/WD5TB-Data/Tim/Pictures/ -type f -print0 | xargs -0 sudo chflags schg

Lock NAS server ¶

On pve:

sudo -u backupmba find /tank/backups/data-helene -type f -print0 | xargs -0 sudo chattr +i
sudo -u backupmbp find /tank/backups/data-tim -type f -print0 | xargs -0 sudo chattr +i

Parity ¶

Make par2 files for all files.

TODO: add exception for .DS_Store and some hidden files.

Create PAR2 ¶

One photo dir:

rm *par2 && par2 c -r1 -q -- "$(basename $(pwd)).par2" *

One photo year dir:

find . -type d -print0 | xargs -0 -I {} -P 4 par2 c -r1 -q -- "{}/{}.par2" "{}/*"

All dirs:

for dir in 20*<12-24>*; do
    cd /Users/tim/Pictures/${dir}; pwd; find . -type d -print0 | xargs -0 -I {} -P 4 par2 c -r1 -q -- "{}/{}.par2" "{}/*"
done

Check / repair files ¶

One photo dir:

par2 v -q -- "$(basename $(pwd)).par2"

One photo year dir:

find . -type d -print0 | xargs -0 -I {} -P 4 par2 v -q -- "{}/{}.par2" | grep "Target.*- missing.\|Target.*- damaged."

All dirs:

for dir in 20*<12-24>*; do
    cd /Users/tim/Pictures/${dir}; pwd; find . -type d -print0 | xargs -0 -I {} -P 4 par2 v -q -- "{}/{}.par2" | grep "Target.*- missing.\|Target.*- damaged."
done

#Adblock #Arduino #Css #Curl #Debian #Diy #Dyndns #E-Mail #Electricity #Epex #ESP8266 #Gandi #Grafana #Heating #Home-Assistant #Home-Improvement #Html #Htpc #Hugo #Influxdb #Ios #Letsencrypt #Linux #Mac #Markdown #Networking #Nextcloud #Nginx #Photography #Proxmox #RaspberryPi #Routeros #Security #Server #Smarthome #Solar #Soverin #Sqlite #Ssh #Transip #Ubuntu #Unix #Vyos #Windows