remote-backups.comremote-backups.com
Contact illustration
Sign In
Don't have an account ?Sign Up

The 3-2-1 backup rule has been gospel for decades: three copies, two media types, one offsite. Solid advice when the biggest threats were disk failures and building fires. But ransomware changed the equation. Modern attacks don't just encrypt production data. They hunt for backups, delete snapshots, and wipe connected storage before dropping the ransom note. The 3-2-1-1-0 strategy adds two numbers that address exactly this gap, and implementing it with Proxmox Backup Server is more straightforward than you'd expect.

Key Takeaways
  • 3-2-1-1-0 means: 3 copies, 2 media types, 1 offsite, 1 air-gapped or immutable, 0 unverified restores.
  • PBS sync jobs with --encrypted-only handle the offsite and immutability requirements.
  • Proxmox Backup Server has no 'immutable backup' flag. Immutability is architectural: separate credentials, restricted permissions, and network isolation.
  • Verification jobs check chunk integrity but don't replace full restore drills.
  • A remote PBS with isolated credentials is a 'logical' air-gap. True air-gap means physically disconnected.
  • The '0' is the most important number. Untested backups are just expensive hope.

What Each Number Means

The 3-2-1-1-0 strategy extends the classic rule to defend against ransomware and credential compromise. Each digit has a specific requirement.

3 copies of your data must exist at all times. The production dataset counts as one. A local Proxmox Backup Server instance is the second. An offsite backup is the third. If any single copy is lost or corrupted, two remain.

2 different media types prevent a single class of hardware failure from taking out multiple copies. Running local backups on the same NVMe array as your VMs doesn't count. Use SSDs for local hot backups and HDDs for offsite storage.

1 offsite copy protects against site-level disasters: fire, flood, theft, or a power surge that takes out everything in the same rack. The offsite copy must be geographically separated, not just in a different room.

1 air-gapped or immutable copy is the ransomware defense. This copy cannot be modified or deleted by anyone with access to your production systems. Not your backup admin, not your monitoring service account, not the attacker who just stole your credentials.

0 errors means every backup in your chain has been verified. Not "it completed successfully." Verified as in: you restored it and the data was intact. Unverified backups are Schrodinger's backups. They might work. You won't know until the worst possible moment.

Why 3-2-1 Is No Longer Enough

What Changed
3-2-1 (Classic)
  • 3 copies of data
  • 2 different media types
  • 1 offsite copy
  • No protection against credential compromise
  • Connected backups assumed safe
3-2-1-1-0 (Modern)
  • 3 copies of data
  • 2 different media types
  • 1 offsite copy
  • 1 air-gapped or immutable copy
  • 0 unverified restores

Ransomware operators study backup tools. They know that most organizations reuse the same root password across their entire Proxmox cluster. They know that backup datastores are often mounted via NFS with shared credentials. They map your infrastructure before they detonate anything.

A typical attack pattern: gain initial access through a phishing email or exposed RDP port, escalate to domain admin, locate backup infrastructure, delete or encrypt all backup data, then deploy the ransomware payload. The entire process takes days or weeks. By the time the ransom note appears, your backups are already gone.

One European hosting provider lost 18 months of customer backups when an attacker compromised their centralized management platform. The backup server used the same LDAP credentials as production. Every snapshot, every replica, every offsite copy accessible from that same credential store was wiped in under four hours.

Shared Credentials = Shared Fate

If your local and offsite Proxmox Backup Server instances share a root password, API token, or LDAP backend, a single credential compromise wipes both. The 3-2-1 rule assumes your backup infrastructure is trustworthy. That assumption no longer holds.

Implementing 3-2-1-1-0 with Proxmox Backup Server

The Three Copies

Copy one is your production data on Proxmox VE. Copy two is your local PBS instance, backing up VMs and containers on a nightly schedule. Copy three is a remote Proxmox Backup Server instance at a separate location, receiving data through PBS sync jobs.

This is the minimum. Some environments add a fourth copy (tape, object storage, or a second offsite location) for critical datasets.

Two Media Types

Your local PBS should run on fast storage. NVMe or SSD arrays give you quick backup windows and fast restores for daily operations. The offsite PBS can run on spinning disks. Restore speed matters less for disaster recovery, and the cost savings on large retention windows are significant.

If your local PBS runs on ZFS with SSDs and your offsite PBS runs on ZFS with HDDs, that satisfies the two-media requirement while keeping the stack consistent.

Offsite Copy

PBS sync jobs replicate datastore contents from one Proxmox Backup Server instance to another. The remote side pulls snapshots from the source on a defined schedule.

For offsite replication, always use encrypted-only mode. This ensures the remote PBS never sees your decryption keys. The remote side stores opaque encrypted chunks that it cannot read, decrypt, or modify.

bash
proxmox-backup-manager sync-job create offsite-daily \
    --store offsite-datastore \
    --remote source-pbs \
    --remote-store local-datastore \
    --encrypted-only true \
    --schedule "daily 02:00" \
    --remove-vanished true
Create sync job on remote PBS
Always use --encrypted-only

Without this flag, unencrypted backups can slip through to the remote side. The remote operator (or attacker) can then read your data. Always enforce client-side encryption with encrypted-only sync.

For a deeper walkthrough of sync job configuration, see PBS Sync Jobs: Offsite Replication Done Right.

Air-Gapped or Immutable

This is where misunderstandings are common.

PBS has no immutable backup flag

There is no checkbox that makes a snapshot undeletable. Immutability in a PBS-based architecture is a property of how you design the system, not a feature you toggle. Protection comes from layering credentials, permissions, and network controls.

Three controls matter for achieving immutability with PBS:

Separate credentials. The offsite Proxmox Backup Server must use completely independent authentication. No shared LDAP, no shared API tokens, no shared root passwords. If an attacker compromises your production Proxmox cluster, they should have zero valid credentials for the offsite PBS.

No delete permissions. The account used for sync jobs should have DatastoreBackup and DatastoreSyncOperator privileges only. Never DatastoreAdmin. An account that can write and sync but not delete provides a meaningful layer of protection.

Network isolation. The offsite PBS should not be reachable from your production network on management ports. Sync traffic can flow through a WireGuard tunnel restricted to the PBS API port. SSH, web UI, and management interfaces should only be accessible from a separate out-of-band network.

True Air-Gap vs. Logical Air-Gap

A true air-gap means physically disconnected. Tape drives pulled from the library. A NAS powered off and locked in a cabinet. No network cable, no power, no attack surface. This is the gold standard, but it requires manual processes and limits your RPO.

A logical air-gap is an always-connected system that is architecturally isolated: different credentials, restricted network access, minimal permissions. A remote Proxmox Backup Server with these controls is logically air-gapped. An attacker who fully compromises your production environment cannot reach or delete the offsite backups without separately compromising the offsite infrastructure.

Most organizations land on a logical air-gap for daily offsite backups and a true air-gap (tape or disconnected disk) for monthly or quarterly archives.

Zero Errors: Verified Restores

PBS verification jobs check chunk-level integrity. They confirm that every chunk referenced by a snapshot exists in the datastore and passes its SHA-256 checksum. Run them regularly:

bash
proxmox-backup-manager verify-job create nightly-verify \
    --store local-datastore \
    --schedule "daily 04:00" \
    --outdated-after 7
Create verification job

But verification is not a restore test. It proves the data is intact on disk. It does not prove you can boot a recovered VM, that your application starts correctly, or that your database is consistent.

Schedule full restore drills quarterly at minimum. Spin up a recovered VM on an isolated network. Check that services start, data is present, and the application functions. Document the results. If you haven't tested a restore, you don't have a backup.

Build restore testing into your runbook

Set a recurring calendar reminder. Pick a random VM each quarter, restore it from offsite, and time the process. This validates your RPO, your RTO, and your team's muscle memory under pressure.

Example Architecture

Loading...
Rendering diagram...
3-2-1-1-0 data flow with Proxmox Backup Server
Example Architecture
Component
Proxmox VE cluster
Local PBS (SSD/NVMe)
Remote PBS (HDD)
Quarterly tape / cold disk
Location
Primary DC
Primary DC
Offsite DC / Cloud
Vault / safe
Role
Production data (copy 1)
Fast local backup (copy 2)
Encrypted offsite sync (copy 3)
True air-gapped archive (copy 4)
Retention
N/A
7 daily, 4 weekly, 3 monthly
14 daily, 8 weekly, 12 monthly
4 quarterly, 1 yearly

Sync jobs run nightly from the remote PBS, pulling encrypted snapshots from the local PBS. The remote Proxmox Backup Server has independent credentials, DatastoreSyncOperator-only permissions, and is reachable only via a dedicated WireGuard tunnel. The quarterly tape or cold disk export is performed manually and stored offsite.

Estimate Your Initial Sync Time
Effective data size70 GB
Estimated transfer time1h 36m

Common Mistakes

  • Same credentials everywhere. If your local and offsite PBS share a root password, API token, or LDAP backend, a single credential compromise wipes both. Use completely independent authentication for the offsite copy.
  • Missing --encrypted-only on sync jobs. Without this flag, unencrypted backups can slip through to the remote side. The remote operator (or attacker) can then read your data. Always enforce client-side encryption with encrypted-only sync.
  • Skipping restore tests. Verification jobs confirm chunk integrity, not application-level recoverability. A backup that passes verification can still produce a VM that won't boot. Test your restores.
  • Offsite PBS on the same network segment. If the offsite PBS management interface is reachable from your production VLAN, it's not air-gapped in any meaningful sense. Use firewall rules, VLANs, or a dedicated WireGuard tunnel that only exposes the sync API endpoint.
  • No garbage collection schedule. PBS datastores grow without regular garbage collection. Orphaned chunks consume disk space and slow verification. Schedule GC weekly and monitor datastore usage.
  • Ignoring the "0" in 3-2-1-1-0. The zero is arguably the most important number. Without verified restores, every other layer is theoretical. Build restore testing into your runbooks, not your "someday" list.

Wrapping Up

The 3-2-1-1-0 strategy isn't complicated to implement with Proxmox Backup Server. You need a local instance, a remote instance with isolated credentials, encrypted-only sync jobs, and a calendar reminder to actually test your restores. The tooling exists. The architecture patterns are proven. The only variable is whether you build it before or after an incident forces your hand.

Need the offsite leg handled for you?

remote-backups.com provides encrypted Proxmox Backup Server targets in EU datacenters with isolated credentials and geo-replication.

View Plans

Frequently Asked Questions

No. PBS has no built-in immutable snapshot flag. Immutability is achieved through architecture: separate credentials, restricted permissions (DatastoreSyncOperator instead of DatastoreAdmin), and network isolation between your production and offsite PBS instances.

A true air-gap is physically disconnected storage (tape, powered-off NAS). A logical air-gap is a connected system that is architecturally isolated: different credentials, restricted network access, and minimal permissions. Most organizations use a logical air-gap for daily offsite backups and a true air-gap for quarterly archives.

Without --encrypted-only, the remote PBS can receive unencrypted backups if client-side encryption isn't enforced at the source. With it enabled, the sync job rejects any unencrypted snapshot, ensuring the remote operator cannot read your data.

No. Verification confirms chunk-level integrity on disk (checksums pass, all referenced chunks exist). It does not prove you can boot a recovered VM, that applications start correctly, or that databases are consistent. You need periodic full restore drills for that.
Bennet Gallein
Bennet Gallein

remote-backups.com operator

Infrastructure enthusiast and founder of remote-backups.com. I build and operate reliable backup infrastructure powered by Proxmox Backup Server, so you can focus on what matters most: your data staying safe.