remote-backups.comremote-backups.com
Contact illustration
Sign In
Don't have an account ?Sign Up

Initial Seed Loading for Remote PBS

Your first offsite backup is always the hardest. Every chunk needs to cross the wire before deduplication can kick in on the remote side. For a 500 GB homelab, that's an afternoon. For 10 TB of client VMs at an MSP, that's potentially weeks of saturated uplinks. This post covers four strategies for getting that initial seed onto a remote Proxmox Backup Server without losing your mind (or your clients' patience).

Key Takeaways
  • The initial PBS sync transfers every chunk — deduplication only helps after the first full copy lands
  • 10 TB at 100 Mbps takes roughly 10 days of continuous transfer
  • Physical drive shipping beats WAN transfer for anything above 5 TB in most scenarios
  • Rate limiting and off-hours scheduling protect production traffic during WAN seeds
  • After the seed, daily incrementals typically move 1-5% of total dataset size
  • Use the same encryption key for seeding and ongoing backups — mismatched keys mean a full resync

The Initial Seed Problem

Proxmox Backup Server stores backups as content-addressed chunks. When the remote datastore already has a chunk, it skips the transfer. That's what makes daily incrementals fast: most chunks already exist. But on day zero, the remote datastore is empty. Every single chunk must be uploaded.

The math is straightforward. A 10 TB dataset at 100 Mbps of sustained throughput takes about 9.5 days. At 50 Mbps, that's 19 days. Factor in PBS protocol overhead (roughly 5-10% on high-latency links) and you're looking at even longer. Our initial seed calculator gives you exact numbers for your scenario, and the TCP throughput calculator shows how latency and window size affect what your link can actually deliver.

For MSPs onboarding new clients, this creates a real problem. You can't tell a client their offsite backups won't be functional for three weeks. And you can't saturate their production uplink for that long either.

The solution depends on your dataset size, available bandwidth, and how quickly you need the offsite copy operational.

Strategy 1: Saturate the WAN

The simplest approach: push everything over the network. This works well when your dataset is under 2 TB or you have a dedicated link with capacity to spare.

Rate Limiting

The proxmox-backup-client supports a --rate flag that caps upload throughput in bytes per second. Use this to protect production traffic during business hours.

bash
proxmox-backup-client backup \
    vm/100.img:/dev/zvol/rpool/data/vm-100-disk-0 \
    --repository remote-pbs:datastore1 \
    --rate 50MB
Backup with rate limiting (50 MB/s cap)

50 MB/s (roughly 400 Mbps) leaves headroom on a gigabit link. Drop this to 12 MB/s (~100 Mbps) if you're sharing a 500 Mbps connection with production services.

Off-Hours Scheduling

Combine rate limiting with cron scheduling to run unrestricted seeds overnight when no one is using the link.

bash
# Full speed overnight (22:00-06:00)
0 22 * * * root /usr/local/bin/pbs-seed.sh --rate 0
# Rate-limited during business hours (06:00-22:00)
0 6  * * * root /usr/local/bin/pbs-seed.sh --rate 12MB
/etc/cron.d/pbs-initial-seed
QoS at the network level

Rate limiting in the PBS client is a blunt instrument. If your firewall supports traffic shaping (pfSense, OPNsense, or Linux tc), mark PBS traffic with DSCP and give it a dedicated queue. This protects latency-sensitive services like VoIP even when the seed runs at full speed.

When WAN Seeding Falls Apart

At 100 Mbps, anything over 2 TB takes more than two days. At 50 Mbps, even 1 TB takes two days. If your client can't tolerate that window, or the link quality causes frequent retries, you need a different approach.

For high-latency links specifically, PBS protocol overhead compounds the problem. Each chunk upload waits for an acknowledgment before the next is sent. Our edge locations solve this with a write accelerator that ACKs chunks locally, but for the initial seed of very large datasets, physical media is often faster.

Strategy 2: Ship Physical Drives

Andrew Tanenbaum said it best: "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." For datasets above 5 TB, shipping a drive is often faster, cheaper, and less disruptive than a WAN transfer.

The Process

  1. Back up locally. Run your Proxmox Backup Server backup jobs against a local PBS instance as you normally would.
  2. Encrypt. If you're using client-side encryption, the chunks on disk are already encrypted. If not, encrypt the drive before shipping (LUKS or similar).
  3. Ship the drive. Send it to your offsite provider via tracked courier.
  4. Provider imports chunks. The provider loads the chunks into your remote datastore.
  5. Configure sync jobs. Once the seed is in place, set up PBS sync jobs for ongoing incremental replication.
No official import tool

Proxmox Backup Server has no built-in command for importing a datastore from a shipped drive. The chunk files and manifest structure can be copied at the filesystem level, but this requires admin access on the destination PBS. When using a managed provider, they handle this step. Ask your provider about their seeding process before shipping hardware.

Encryption Matters

If you ship unencrypted drives, your backup data is readable by anyone who intercepts the package. Always encrypt.

Client-side encryption in PBS is the cleanest approach. Chunks leave your infrastructure encrypted with your key. The remote PBS stores them without being able to read them. This is the same encryption used for ongoing sync jobs with --encrypted-only, so the transition from seed to incremental sync is seamless.

Encryption key consistency

The encryption key used for the physical seed must match the key used for ongoing backups. If you generate a new key after seeding, the remote PBS treats every chunk as new, and you're back to a full transfer. Store your encryption key securely and use it consistently across all backup jobs.

Cost and Speed Comparison

A 16 TB drive costs roughly €250. Overnight shipping within Europe runs €30-50. Total: under €300 and your seed is loaded within 2-3 business days. Compare that to tying up a 100 Mbps link for 15+ days.

Strategy 3: Datacenter Seed Station

If your offsite provider is within driving distance, or if they offer a seed station service, you can skip shipping entirely.

Option A: Bring your hardware. Physically transport a server or NAS to the datacenter. Connect to the provider's network at 10 Gbps. Seed at line speed. 10 TB at 10 Gbps takes about 2.5 hours. Retrieve your hardware when done.

Option B: Provider-hosted seed station. Ship a drive to the provider. They mount it, import the chunks at local disk speed, and either return or securely wipe the drive.

This approach is common with hosting providers who already have cage or rack space for client equipment. It eliminates WAN bottlenecks entirely and gets you operational in hours rather than days.

Strategy 4: Incremental Migration

Not every dataset needs to land on the remote PBS at once. If you're managing a client environment with dozens of VMs, prioritize and stage the migration.

Triage by Importance

Start with the VMs that matter most: domain controllers, database servers, and anything the client can't rebuild from a template. These are often the smallest VMs too, so the initial seed completes quickly.

bash
# Back up domain controller and DB server first
proxmox-backup-client backup \
    vm/100.img:/dev/zvol/rpool/data/vm-100-disk-0 \
    vm/101.img:/dev/zvol/rpool/data/vm-101-disk-0 \
    --repository remote-pbs:datastore1

# Add remaining VMs over the next week
proxmox-backup-client backup \
    vm/102.img:/dev/zvol/rpool/data/vm-102-disk-0 \
    vm/103.img:/dev/zvol/rpool/data/vm-103-disk-0 \
    --repository remote-pbs:datastore1 \
    --rate 25MB
Seed critical VMs first

Deduplication Across VMs

Once the first few VMs are seeded, deduplication starts working in your favor. VMs running the same OS share identical base chunks. The second Windows Server 2022 VM transfers a fraction of the data because most of its OS chunks already exist on the remote datastore. If you're an MSP managing multiple tenants, this effect compounds across clients with similar environments.

Run Existing Backups in Parallel

Keep your existing backup solution running during the seed period. The remote PBS seed doesn't replace your local backups. It runs alongside them. Once the seed completes and sync jobs are verified, you can retire the old offsite method.

Bandwidth Planning

Use these estimates for planning conversations with clients. All assume sustained throughput with no competing traffic.

Estimated Seed Duration by Dataset Size
Dataset Size
1 TB
5 TB
10 TB
25 TB
50 TB
50 Mbps
1.9 days
9.5 days
19 days
47 days
94 days
100 Mbps
22 hours
4.7 days
9.5 days
24 days
47 days
250 Mbps
9 hours
1.9 days
3.8 days
9.5 days
19 days
500 Mbps
4.5 hours
22 hours
1.9 days
4.7 days
9.5 days
1 Gbps
2.2 hours
11 hours
22 hours
2.3 days
4.7 days
Protocol overhead

These numbers assume raw throughput. PBS protocol overhead adds 5-10% on high-latency links due to per-chunk acknowledgment waits. On low-latency links (under 10 ms), overhead is negligible. Factor this into your estimates, especially for WAN seeds crossing continents.

For exact numbers tailored to your setup, plug your values into the initial seed calculator.

After the Seed

Once the initial dataset lands on the remote Proxmox Backup Server, daily operations become dramatically easier.

Incrementals are small. A typical daily change rate for VM workloads is 1-5% of the total dataset. A 10 TB environment generates 100-500 GB of new chunks per day. At 100 Mbps, that's 2-11 hours, well within an overnight backup window.

Deduplication compounds. Every new client VM that shares OS-level chunks with existing backups transfers less data. The second, third, and tenth client seed faster than the first.

Sync jobs handle the rest. Configure PBS sync jobs to pull new snapshots on a schedule. The remote PBS only downloads chunks it doesn't already have. Set pruning and garbage collection policies independently on source and destination to match your retention requirements.

Monitor the pipeline. Set up backup monitoring to catch failed syncs early. A missed sync is recoverable if you catch it within your retention window. A week of missed syncs might mean another seed.

Common Mistakes

Mismatched encryption keys. Generating a new key after seeding forces a complete resync. Use the same key from day one.

Seeding without testing restore. Verify you can restore from the remote datastore before telling the client they're protected. Run a restore test against the remote PBS as soon as the seed completes.

Ignoring retention overlap. If your local PBS prunes snapshots before the sync job pulls them, those snapshots never reach the remote. Schedule syncs to run after backups complete but before pruning.

Saturating the client's link. Nothing kills an MSP relationship faster than dropping VoIP calls because you're running an unthrottled seed at 2 PM on a Tuesday. Always rate-limit during business hours.

Wrapping Up

The initial seed is a one-time cost. Get it right, and every day after is incremental, fast, and automatic. For datasets under 2 TB, WAN seeding with rate limiting works fine. Above 5 TB, physical media or a datacenter seed station saves days or weeks. Incremental migration lets you bring clients online VM-by-VM without waiting for the entire dataset to transfer.

Pick the strategy that matches your bandwidth, timeline, and client expectations. The goal is the same in every case: get that first full copy onto the remote Proxmox Backup Server so deduplication and sync jobs can take over.

Skip the seed logistics

remote-backups.com offers managed seeding with edge locations across Europe. Ship a drive or seed at our datacenter. We handle the import and get your offsite backups running in days, not weeks.

View Seeding Options

Technically yes. The chunks and manifests are regular files. But PBS expects specific directory structures and metadata. Copying raw files requires admin access on the destination and careful validation. There is no official import command. Managed providers handle this as part of their seeding service.

Yes. Chunks are content-addressed. Once a chunk exists on the remote datastore (from the seed), it will never be uploaded again, regardless of which VM or backup job references it.

PBS tracks which chunks have been uploaded. When you restart the backup or sync job, it resumes from where it left off. You don't lose progress, but you should monitor for repeated failures that might indicate a network issue.

PBS already stores chunks in a compressed format. Additional compression on top adds CPU overhead with minimal space savings. Encrypt the drive instead of compressing it.

Run a verification job on the remote PBS after importing the seed data. This checks chunk integrity and ensures nothing was corrupted during transfer or import. Then do a test restore to confirm end-to-end recoverability.
Bennet Gallein
Bennet Gallein

remote-backups.com operator

Infrastructure enthusiast and founder of remote-backups.com. I build and operate reliable backup infrastructure powered by Proxmox Backup Server, so you can focus on what matters most: your data staying safe.