Back Up Linux Servers to S3 with restic and rclone

Not every system you run speaks the Proxmox Backup Server protocol. A VPS at another provider, a database host, a web server with uploads, a NAS: these often need a simple offsite copy, and the tool of choice is usually restic or rclone. Both tools speak S3. This guide shows how to connect them to an S3-compatible bucket, run backups on a schedule, set a retention policy, and test a restore.

The examples use Buckets, our S3-compatible object storage in Frankfurt and Düsseldorf. The commands work the same way with any S3-compatible endpoint. Only the endpoint, region, and credentials change.

restic or rclone: Which Tool for Which Job

The two tools solve different problems. Many servers use both.

If you only need one, pick restic for backups. Pick rclone when other people or programs must read the files in the bucket directly. For a deeper comparison of restic with Borg, read Borg vs Restic.

Step 1: Create a Bucket and Get the Credentials

In the dashboard, go to Buckets > Create Bucket. Choose a region (Frankfurt or Düsseldorf), a size between 100 GB and 20 TB, a bandwidth limit (100, 250, or 500 Mbit/s included, or 1 Gbit/s as a paid add-on), and optionally autoscaling.

When the bucket is ready, the bucket page shows five values, each with a copy button:

• Endpoint, for example https://fra1.my-s3.com
• Region, for example fra1
• Bucket, the bucket name you use in paths
• Access Key ID
• Secret Key

The credentials are scoped to this one bucket. A leaked key on one server cannot read or delete data in your other buckets.

The commands below use placeholders. Replace <bucket>, <access-key-id>, and <secret-key> with your values. If your bucket is in Düsseldorf, use the endpoint and region that your dashboard shows.

Step 2: Back Up with restic

Store the credentials in a root-only file

Do not put secrets in shell history or in a crontab. Put them in a file that only root can read. First, create the folder and the password file:

mkdir -p /etc/restic
chmod 700 /etc/restic
openssl rand -base64 32 > /etc/restic/password
chmod 600 /etc/restic/password

Then create /etc/restic/env with your values and run chmod 600 /etc/restic/env. The same file works in your shell and in systemd:

AWS_ACCESS_KEY_ID="<access-key-id>"
AWS_SECRET_ACCESS_KEY="<secret-key>"
AWS_DEFAULT_REGION="fra1"
RESTIC_REPOSITORY="s3:https://fra1.my-s3.com/<bucket>"
RESTIC_PASSWORD_FILE="/etc/restic/password"
RESTIC_CACHE_DIR="/var/cache/restic"

Initialize the repository and run the first backup

set -a; . /etc/restic/env; set +a
restic init
restic backup /etc /home /var/www \
  --exclude-caches \
  --tag daily
restic snapshots

restic init creates the repository structure in the bucket. The first backup uploads everything. The next runs upload only new and changed chunks, so they are much smaller.

If a restic command fails with a bucket-lookup or DNS error, add -o s3.bucket-lookup=path. This forces path-style requests (https://endpoint/<bucket>), which the endpoint expects.

Back up databases as a stream

For databases, back up a consistent dump, not the live data files. restic can read the dump from stdin, so no temporary file is necessary:

set -a; . /etc/restic/env; set +a
sudo -u postgres pg_dumpall \
  | restic backup --stdin --stdin-filename postgres.sql --tag db

Retention and integrity checks

restic keeps every snapshot until you tell it to forget. A common policy is 7 daily, 4 weekly, and 6 monthly snapshots:

set -a; . /etc/restic/env; set +a
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune
restic check --read-data-subset=5%

--prune deletes data that no snapshot needs any more. restic check --read-data-subset=5% downloads and verifies a random 5% of the data on each run, so over many runs it reads most of the data. The retention logic is similar to the PBS keep rules; see How PBS keep-* prune rules work if you want the details.

Step 3: Schedule It with systemd

A systemd timer is more reliable than cron: it logs to the journal, and Persistent=true runs a missed job after a reboot.

[Unit]
Description=restic backup to S3
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
EnvironmentFile=/etc/restic/env
CacheDirectory=restic
ExecStart=/usr/bin/restic backup /etc /home /var/www --exclude-caches --tag daily
ExecStartPost=/usr/bin/restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

[Unit]
Description=Daily restic backup

[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=30m
Persistent=true

[Install]
WantedBy=timers.target

Enable the timer and check it:

systemctl daemon-reload
systemctl enable --now restic-backup.timer
systemctl list-timers restic-backup.timer
journalctl -u restic-backup.service -n 50

A failed backup that nobody sees is the same as no backup. Send the result of the service to your monitoring. The post on backup monitoring and alerting shows some options.

Step 4: Mirror Files with rclone

Use rclone when the files in the bucket must stay readable as normal files. Add a remote to ~/.config/rclone/rclone.conf. The bucket page in the dashboard shows this block ready to copy, with the remote name my-s3. This guide renames it to rb-fra1:

[rb-fra1]
type = s3
provider = Other
access_key_id = <access-key-id>
secret_access_key = <secret-key>
endpoint = https://fra1.my-s3.com
region = fra1

Test the connection, then sync:

rclone lsd rb-fra1:<bucket>
rclone sync /srv/media rb-fra1:<bucket>/current \
  --backup-dir rb-fra1:<bucket>/archive/$(date +%F) \
  --transfers 8 \
  --progress

rclone sync makes the destination equal to the source. It also deletes files in the bucket that you deleted on the server. --backup-dir moves changed and deleted files into a dated folder first, so a mistake on the server does not remove the only copy. Clean up old archive folders from time to time, for example with rclone purge.

If the data must be encrypted before it leaves the server, run rclone config and add a crypt remote on top of rb-fra1:<bucket>/crypt. rclone then encrypts file contents and names on upload. Keep the crypt password safe; the same rule as for restic applies.

Step 5: Test a Restore

Do this now, not on the day you need it.

set -a; . /etc/restic/env; set +a
restic snapshots
restic restore latest --target /tmp/restore-test --include /etc/nginx
diff -r /etc/nginx /tmp/restore-test/etc/nginx

For a quick look at an old snapshot, restic mount /mnt/restic gives a browsable folder of every snapshot (it needs FUSE). With rclone, copy the files back: rclone copy rb-fra1:<bucket>/current/photos /tmp/restore-test.

Put the restore test on a calendar. Our guide to restore testing and DR drills gives a simple plan.

Sizing, Cost, and Limits

Buckets cost €6 per TB per month (net), metered hourly on the provisioned bucket size. A 500 GB bucket costs €3 per month, full or empty. There are no request fees.

Two consequences for backups:

• Plan headroom for prune. restic writes new packs before it deletes old ones. Keep some free space, or enable autoscaling. Autoscaling grows the bucket by 100 GB when usage crosses your threshold (90% by default) and sends you an email.
• Pick the bandwidth for your backup window. At 100 Mbit/s, 1 TB takes about 22 hours to upload. At 500 Mbit/s, it takes about 4.5 hours. At 1 Gbit/s (paid add-on), it takes about 2.2 hours. On subscription buckets, you can change the limit later in the bucket settings.

Objects are stored redundantly inside one region. They are not copied to a second region. If your policy needs two locations (the 3-2-1 rule), create a second bucket in the other region and sync between them. Add a second rclone remote (here rb-dus1) with the endpoint, region, and keys from the bucket page of the new bucket:

rclone sync rb-fra1:<bucket> rb-dus1:<second-bucket> --transfers 8

A restic repository is a set of normal objects, so rclone can copy it 1:1. The copy is a complete repository; restic can read it with the same password. Run the sync when no backup or prune is active, so the copy is consistent.

Conclusion

restic and rclone cover the servers that PBS does not: VPS hosts, database servers, and file shares. restic gives you encrypted, deduplicated snapshots with retention. rclone gives you a plain mirror that other tools can read. Both need only an endpoint, a region, and a key pair.

Start with one server. Create a bucket, run the first restic backup, schedule it, and test a restore the same day. Then add the next server. Compare the options on the pricing page, or create your first bucket from the Buckets page.