Automated encrypted backups for $18/yr

return to homepage

 

This is a tutorial for creating automated, versioned, deduplicated and encrypted remote backups on a linux machine. Let's define our assumptions and all 'em big words.

Assumptions

• You have a strategy regarding what to store.

  • If you don't, a good starting point is the borg docs on strategy.

• You want to backup less than 100GB.

  • If you need to store 2TB of movies, code, pictures, ... then $18 won't be enough. (Wasabi, Backblaze B2, or AWS S3 may be more affordable).

• You how how to use a unix command line, know how to ssh and how to edit .conf files.

Specifications

Here are some things we want to achieve:

1. backups happen automatically (we will use systemd to schedule them. But anacron or cron work too)
2. We don't want to delete a file in our system, then back it up, the file gets deleted in our backup. ... but oops! A day later we realize we needed that file. This often happens with Dropbox or with similar systems. (note that Dropbox Rewind lets you find previous versions of the file up to 30 or 180 days).
3. We don't want to upload all the files every time we do a backup. We only want to upload changes.
4. our backups should be encrypted by our local computer. We have the key, no one else! (unlike Dropbox, Google Drive, etc ... who can read your files)

 

Therefore our backups are:

• automated (with systemd)
• versioned (we can rewind to the state of a file or of the whole system to a week ago or 2 years ago)
• deduplicated (we don't upload duplicated data, only changes)
• encrypted

 

To achieve this we will use restic, and backup to rsync.net using their special discount for restic.

Strategy

The general idea is simply to run this command regularly:

restic -r where/to/save backup folders/to/backup 

followed by the maintenance operations of restic forget and the occasional restic prune.

Now, with the full details, these commands can become too wordy:

​x
# backup
restic -r sftp:XXX@host.net:/path/nameofrepo backup --verbose --password-file SUPERSECRET --tag sometag --exclude="*.c" --exclude-file=excludes.txt -files-from /tmp/files_to_backup /tmp/some_additional_file
​
# forget
restic forget --verbose --tag systemd.timer --group-by "paths,tags" --keep-daily $RETENTION_DAYS --keep-weekly $RETENTION_WEEKS --keep-monthly $RETENTION_MONTHS --keep-yearly $RETENTION_YEARS
​

 

So we are going to separate it into 3 files:

• a general configuration file
• a list of files / directories to include
• a list of files / directories to exclude

 

The Nitty-Gritty

Let's do it:

xxxxxxxxxx
cd ~
mkdir .config/backup/

There, create 3 files:

online-restic-backup.conf

xxxxxxxxxx
BACKUP_INCLUDES="--files-from='/home/youruser/.config/backup/restic-includes'"
BACKUP_EXCLUDES="--exclude-file='/home/youruser/.config/backup/restic-excludes'"
RETENTION_DAYS=7
RETENTION_WEEKS=4
RETENTION_MONTHS=6
RETENTION_YEARS=3
RESTIC_REPOSITORY="sftp:XXXXX@ab-x331.rsync.net:reponame"
RESTIC_PASSWORD="something very long horse staple you know the drill"

Note:

using the env command, a system() call or using inline shell scripts (e.g. RESTIC_PASSWORD=password restic …) might expose the credentials in the process list directly and they will be readable to all users on a system. Using export in a shell script file should be safe, however, as the environment of a process is accessible only to that user. Please make sure that the permissions on the files where the password is eventually stored are safe (e.g. 0600 and owned by root).

The repository sftp:XXXXX@ab-x331.rsync.net:reponame will be different for you, of course. But we will come back to it.

restic-includes

xxxxxxxxxx
# files to include
​
# dotfiles
/home/youruser/.bashrc
/home/youruser/.bash_aliases
​
# some config files
/home/youruser/.config/backup
/home/youruser/.config/systemd
​
# keybase after it was acquired by zoom
/keybase/team/thisthat
​
# etc
​

 

restic-excludes

xxxxxxxxxx
# no videos in this directory
/keybase/private/*/*.avi
​
# not this subdir 
/home/youruser/Documents/cats
​
# skip this dir
/keybase/private/me,kgb

or check the docs for excluding.

 

Initial Steps

Before we keep going, do the installations and signing-up.

Install restic: https://restic.readthedocs.io/en/latest/020_installation.html

sign up at: https://www.rsync.net/products/restic.html

you will get a user with format NNNNN and a server with format ab-xNNN.rsync.net or something similar.

Your restic repository will now live at: sftp:XXXXX@ab-x331.rsync.net:reponame, so make sure you update your config file above. Then follow their instructions to get ssh access with a key.

 

You are now ready to initialize the repo. Chose a name for it, here myrepo and run in your local shell:

xxxxxxxxxx
restic -r sftp:XXXXX@ab-x331.rsync.net:myrepo init

Chose your password and store it in your config file.

You can check that it works, for example with a small file:

xxxxxxxxxx
restic -r sftp:XXXXX@ab-x331.rsync.net:myrepo --verbose backup .some_dotfile

followed by:

xxxxxxxxxx
restic -r sftp:XXXXX@ab-x331.rsync.net:myrepo snapshots

 

Scheduling

Now we need to tell our computer to do this regularly.

xxxxxxxxxx
cd ~
mkdir .config/systemd/user/

note that user is not your username, it's the actual string user.

inside that directory create this file:

restic-backup.service

xxxxxxxxxx
[Unit]
Description=Restic backup service (on rsync.net)
[Service]
Type=oneshot
# the command that will be run by this service:
ExecStart=restic backup --verbose --tag systemd.timer $BACKUP_EXCLUDES $BACKUP_INCLUDES
​
# the command to run after that:
ExecStartPost=restic forget --verbose --tag systemd.timer --group-by "paths,tags" --keep-daily $RETENTION_DAYS --keep-weekly $RETENTION_WEEKS --keep-monthly $RETENTION_MONTHS --keep-yearly $RETENTION_YEARS
# the configuration file with env. variables:
EnvironmentFile=/home/youruser/.config/backup/online-restic-backup.conf

as well as:

restic-backup.timer

xxxxxxxxxx
[Unit]
Description=Backup to (rsync.net) daily
RefuseManualStart=no        # Allow manual starts
RefuseManualStop=no         # Allow manual stops
​
[Timer]
# every day at 9:30pm
OnCalendar=*-*-* 21:30:00
# Execute job if it missed a run due to machine being off
Persistent=true
# File describing job to execute
Unit=restic-backup.service
​
[Install]
WantedBy=timers.target

Ok. All the infrastructure is set. We now need to start it:

xxxxxxxxxx
# reload systemd
systemctl --user daemon-reload
​
# manual start/stop
systemctl --user start ssd-sp-restic-backup # (single run)
systemctl --user stop ssd-sp-restic-backup # (stop the backup, if needed)
​
# start timer
systemctl --user enable --now ssd-sp-restic-backup.timer # (enable & start timer)

 

Now, the backup should happen daily at the chosen time, or at the next available time if the computer is off. You can do the same process to prune your restic repos. You will create a restic-prune.service and a restic-prune.timer , with the only major difference being in:

xxxxxxxxxx
ExecStart=restic prune --verbose

which calls the same configuration file.

 

I have also set this backup system to make an encrypted backup to my external hard drives. To make it easier to create manual backups, I also created a bash script as seen below:

backits.sh

xxxxxxxxxx
#!/bin/bash
​
# display green comments in the command line print
green=$(tput setaf 2)
normal=$(tput sgr0)
​
# help
if [ "$1" == "-h" ]; then
    printf "\nBacks up an encrypted copy of certain files\n"
    printf "to a restic repository (deduplicated, versioned repo)\n\n"
​
    printf "backits -free ${green} -> backup to local FREECOM SSD drive${normal}\n"
    printf "backits -sp ${green} -> backup to local SP ARMOR SSD drive${normal}\n"
    printf "backits -o ${green} -> backup online to rsync.net ${normal}\n\n"
​
    printf "backits -free --prune ${green} -> prune local FREECOM SSD repo${normal}\n"
    printf "backits -sp --prune ${green} -> prune local SP ARMOR SSD repo${normal}\n"
    printf "backits -o --prune ${green} -> prune online rsync.net repo ${normal}\n\n"
​
    printf "backits -o --snap ${green} to display online shapshots (NEEDS REPO PASSWORD)${normal}\n"
    printf "backits -o --log ${green} to show systemd log${normal}\n"
fi
​
​
# Login to keybase backup user
current_user=$(keybase whoami)
if [ ${current_user} != "keybaseuser1" ]
   then
       keybase login -s keybaseuser1 # -s for switch user
       sleep 1
fi
​
if [ ${current_user} != "keybaseuser1" ]
   then # the above didn't work
       exit 1
fi
​
​
# do restic backups
if [ "$1" == "-free" ]; then
    if [ "$2" == "--prune" ]; then
    printf "\n prunning FREECOM ssd drive repo\n"    
    systemctl --user start ssd-free-prune
    else
    printf "\n saving to FREECOM ssd drive\n"    
    systemctl --user start ssd-free-restic-backup
    fi
​
fi
​
if [ "$1" == "-sp" ]; then
    if [ "$2" == "--prune" ]; then
    printf "\n prunning SP ARMOR ssd drive repo\n"    
    systemctl --user start ssd-sp-prune
    else
    printf "\n saving to SP ARMOR ssd drive\n"    
    systemctl --user start ssd-sp-restic-backup
    fi
​
fi
​
if [ "$1" == "-o" ]; then
    if [ "$2" == "--prune" ]; then
    printf "\n prunning rsync.net repo\n"    
    systemctl --user start online-prune
    elif [ "$2" == "--snap" ]; then
    restic -r sftp:XXXXX@ab-x331.rsync.net:reponame snapshots
    elif [ "$2" == "--log" ]; then
    journalctl --user --unit online-restic-backup.service
    else
    printf "\n saving to rsync.net \n"    
    systemctl --user start online-restic-backup
    fi
fi
​
​
​

 

Of course, we haven't talk about restoring your backups, but restic makes it very easy. To check the consistency of your backup:

xxxxxxxxxx
restic -r restic-repo check

To fully restore:

xxxxxxxxxx
restic -r restic-repo restore 79766175 --target /tmp/restored

But much more granular recovery of files and repos can be done.

 

 

Useful links:

https://fedoramagazine.org/automate-backups-with-restic-and-systemd/ https://jeetblogs.org/post/you-are-only-as-good-as-your-next-resurrection-part-3/

https://jeetblogs.org/post/you-are-only-as-good-as-your-next-resurrection-part-1/

https://jeetblogs.org/post/scheduling-jobs-with-systemd/

systemd time format

fixing lock errors

return to homepage