Multi-Site PBS Architecture for MSPs

You started with one PBS instance and a handful of clients. Now you have thirty. The flat single-instance model that worked at five clients is groaning under the weight of cross-client noise, retention conflicts, and a permissions matrix nobody fully understands anymore. What follows is the architecture pattern MSPs settle on once a shared box stops scaling: hub-and-spoke, with deliberate namespace and sync design.

The Problem with One PBS for Everything

Run one PBS box for every client and the seams show fast. A failing verify job from one client buries notifications from twenty others. A backup window blowout on a noisy client starves the others competing for the same disk IO. Permission drift makes it impossible to prove tenant isolation during audits. And restore tests slow to a crawl because the GC and verify load never lets up.

Namespaces help with logical separation, but they are a tenancy boundary, not a locality boundary. Your client in Frankfurt still has to push backups over WAN to your London Proxmox Backup Server. That's fine until WAN flaps or the client's DR plan calls for sub-hour local restore.

The model that scales is hub-and-spoke. One central PBS as the system of record for offsite. Optional smaller PBS instances at client sites for local restore performance. Pull from the hub so client networks stay closed.

Hub-and-Spoke: The MSP Reference Architecture

The hub is your single point of offsite truth. It lives in your datacenter or on a managed offsite endpoint like remote-backups.com. Every client backup eventually lands here. Your retention rules, your verify schedule, your monitoring, all centralised.

Spokes are optional. A spoke is a smaller Proxmox Backup Server running at a client site, sized for that client's working set with maybe a week of retention. Its job is local restore speed. When a client loses a VM at 14:00, they do not want to wait for a 200GB pull over a 100Mbit uplink before they can start a restore.

Sync direction matters. Pull mode at the hub is the right default for MSPs. The hub initiates the connection, holds the credentials, and the client site never needs an outbound API token to your infrastructure. If a client environment is compromised, the attacker cannot reach back into your backup pool with stolen credentials.

Namespace Design at the Hub

One top-level namespace per client. Always. Never collapse two clients into the same namespace, even tiny ones, because the moment you do you have broken the cleanest mental model for billing, ACLs, and offboarding.

Below the client namespace, sub-namespaces split by site or workload type. The layout should be obvious to anyone reading the datastore for the first time:

main/
├── client-acme/
│   ├── site-hq/
│   └── site-dr/
├── client-globex/
│   ├── windows-vms/
│   └── linux-vms/
└── client-initech/
    └── default/

This shape gives you per-client ACLs at the top level and per-site or per-workload scoping below. When a client wants read-only access to their own backups, you scope their token to their namespace and nothing else leaks.

# Create the client tree
proxmox-backup-manager namespace create main client-acme
proxmox-backup-manager namespace create main client-acme/site-hq
proxmox-backup-manager namespace create main client-acme/site-dr

# Read-only user for the client
proxmox-backup-manager user create acme-readonly@pbs \
    --password 'change-me'

# Scope ACL to the client's namespace only
proxmox-backup-manager acl update \
    /datastore/main/client-acme \
    acme-readonly@pbs \
    --role DatastoreAudit \
    --propagate true

# Generate an API token for automation
proxmox-backup-manager user generate-token \
    acme-readonly@pbs readonly

DatastoreAudit lets the client see snapshot lists and metadata without pulling chunk data or deleting anything. If you want them to be able to restore, use DatastoreReader instead. Never give a client DatastoreAdmin on the hub.

Sync Scheduling and Bandwidth Control

The default mistake: every client syncs at 02:00. Your hub disk goes flat for two hours, your WAN saturates, and the verify jobs queue up behind the sync.

Stagger. Spread sync windows across the night so no two large clients overlap. Smaller clients can share a window. Use the schedule field on each pull job:

proxmox-backup-manager sync-job create acme-pull \
    --store main --ns client-acme \
    --remote acme-spoke --remote-store local \
    --schedule "02:00" --rate-in 80MB

proxmox-backup-manager sync-job create globex-pull \
    --store main --ns client-globex \
    --remote globex-spoke --remote-store local \
    --schedule "02:30" --rate-in 80MB

proxmox-backup-manager sync-job create initech-pull \
    --store main --ns client-initech \
    --remote initech-spoke --remote-store local \
    --schedule "03:00" --rate-in 40MB

rate-in should be lower than your link's burst capacity, especially for clients who share an uplink with their production traffic. Better to take six hours and not page anyone than two hours and trigger a network alert at the client.

Retention is the tricky part. The temptation is to set hub retention identical to spoke retention. Resist it.

Monitoring a Multi-Site Fleet

PBS has a task log. It is adequate for one client. At thirty clients it is noise, and you will miss things.

Get Prometheus and Grafana in place before client count five, not client count fifty. Proxmox Backup Server exposes a /metrics endpoint covering datastore usage, GC stats, sync job state, and verify job state. Scrape every hub and every spoke. Build per-client dashboards keyed on the namespace label.

What to alert on:

• Backup absence: no successful backup in a namespace for over 25 hours (daily-tier clients).
• Verify job failure on any namespace, ever. Page on this.
• Sync lag greater than 24 hours between spoke and hub.
• Datastore usage above 85 percent on hub or spoke.
• GC job has not completed in 7 days.

Do not alert on individual snapshot completion. Alert on the absence of backup activity. That is the signal that actually matters at scale, because successful backups are normal and silence is the anomaly.

Sync jobs drop their state into the task log and the exporter exposes it. If a sync lags overnight, you want to know by 09:00, not 09:00 the next day when the client opens a ticket.

Wrapping Up

The MSP architecture pattern is boringly consistent once you have built it twice: hub-and-spoke, pull mode, namespace per client, retention tiered by SLA, fleet monitoring in Prometheus. Build for thirty clients on day one even if you have five. The migration from a flat Proxmox Backup Server to a hub-and-spoke layout is painful, and the longer you wait the more snapshot history you have to move.

The hub is the part you do not want to operate yourself. Hardware, replacement disks, capacity planning, offsite seismic separation, EU data residency. Either own a second datacenter or rent the hub from someone who already does.