PBS Wire Protocol: What's On The Wire

Every backup tool moves bytes from A to B somehow. Most of them pick a transport off the shelf: rsync over SSH, restic over SFTP or S3, Veeam over SMB, Bareos over its own TCP framing. Proxmox Backup Server picks none of those. When proxmox-backup-client talks to a PBS host, it speaks a custom binary protocol layered on top of HTTP/2 over TLS, and that choice has consequences for how you deploy, monitor, and debug it.

Why Not S3, NFS, or rsync?

PBS could have shipped on top of an existing protocol. The team picked none of them. The reasons come down to three properties of a backup workload that off-the-shelf protocols handle badly.

Chunk existence checks are the hot path

A PBS backup of a 100 GB VM does not upload 100 GB. It splits the disk into roughly 4 MB content-defined chunks and hashes each one. When the backup session opens, the server hands the client a known-chunks list for the relevant prior snapshot. The client diffs its computed hashes against that set and only ships the chunks the server does not already have. For an incremental backup, 95 percent or more of those chunks are already present and never cross the wire. Avoiding a per-chunk synchronous round trip is the single biggest determinant of total backup wall time.

S3 can answer "do you have this hash" with a HEAD request, but you pay a full TLS round trip per chunk unless you pipeline aggressively, and you still pay HTTP request overhead. NFS has no concept of "do you have this content" because NFS is a filesystem protocol, not a content-addressable one. rsync's rolling-hash trick works for one file but does not scale across millions of chunks shared between snapshots.

Atomic upload plus index registration

A snapshot in PBS is not just a pile of chunks. It is a set of index files (.fidx for fixed-size disks, .didx for dynamic data) that reference chunks by hash, plus a signed manifest. Uploading a chunk and registering it in an index has to be atomic from the snapshot's point of view: either the snapshot is complete and visible, or it does not exist. S3 multipart upload does not model this. NFS would make it racy unless you build a locking layer on top.

Resumability and session state

A backup that dies halfway through must not leave a corrupt half-snapshot in the datastore. PBS solves this with a server-side backup session. The session owns all in-progress uploads. If the client disconnects or the process dies, the session times out and everything in it is discarded. Only finish on a valid session promotes the snapshot into the visible namespace. That model is hard to bolt onto a stateless object protocol.

HTTP/2 Underneath

Once you accept that PBS needs its own protocol, the next question is what to layer it on. HTTP/2 is the answer, and it is the right one.

The transport is TLS 1.2 or 1.3, depending on what both ends negotiate. ALPN advertises h2 during the handshake. If the server cannot negotiate h2, the connection fails. There is no HTTP/1.1 fallback path in the client.

HTTP/2 matters because of stream multiplexing. A single TCP connection carries many independent streams, each with its own flow control. The PBS client opens one connection and runs many parallel chunk uploads as separate streams. You get the parallelism of multiple TCP sockets without the connection-setup tax, and you keep one TLS session for the whole backup. Head-of-line blocking still exists at the TCP layer, but it never bites at the request level.

API endpoints sit at predictable paths. The general API lives under /api2/json/ (JSON-encoded) and /api2/extjs/ (the variant the web UI consumes). Everything backup-specific lives under /api2/json/backup/. The same HTTP/2 connection carries JSON control messages and raw binary chunk uploads. Control endpoints return JSON. Chunk endpoints accept opaque binary bodies and return small JSON responses indicating whether the chunk was new or already present.

Authentication: Tickets vs Tokens

PBS supports two parallel authentication systems. They feed the same authorisation layer (ACLs against resource paths), but they exist for different use cases.

Tickets

Tickets are the PVE-style cookie-and-CSRF flow. The client POSTs a username, password, and realm to /api2/json/access/ticket. The server validates the credentials, signs a ticket containing the user and an expiry, and returns it along with a CSRF prevention token. The ticket is good for two hours and gets renewed on use. Browsers store it as a cookie. The CSRF token goes into a request header for any write operation.

This is what humans use. The web UI lives on tickets. So does any short-lived interactive session, like proxmox-backup-manager invoked at the shell on the PBS host itself. If you script against PBS with curl from your laptop, you can grab a ticket and use it for a couple of hours, but it is not the right tool for automation.

API tokens

API tokens are the right tool for automation. You create a token under a specific user, give it a name, and the server hands you a secret once. The complete credential is user@realm!tokenname:secret. You scope what the token can do with ACLs on the underlying user, and you can revoke a token without touching the user account.

The client presents a token in the Authorization header.

POST /api2/json/backup HTTP/2
Host: pbs.example.com
Authorization: PBSAPIToken=backup@pbs!nightly:8c2a4f9b-1e6d-4a73-9c11-2f8d3e5a7b6c
Content-Type: application/json
Content-Length: 142

The server validates token requests against the signing key in /etc/proxmox-backup/authkey, checks the token has not expired (tokens can have an optional expiry), and then runs the normal ACL check against the resource path. proxmox-backup-client in cron, automation in Ansible, and anything backing up from a PVE host should all use tokens.

If you want a deeper dive into hardening these credentials and the encryption keys that pair with them, see our notes on PBS client-side encryption.

Anatomy of a Backup Session

A backup is a state machine on the server. Walking through one snapshot, end to end, makes the protocol shape obvious.

1. The client opens a session by hitting /api2/json/backup with the target namespace, group, backup type (vm, ct, or host), backup-id, and a timestamp. This endpoint performs an HTTP/2 protocol upgrade: the response switches the connection into a custom session where subsequent requests address endpoints by relative name, not host-root path. The server creates the session, allocates a temporary directory under the datastore, and returns a session identifier.
2. Early in the session, the server provides the known-chunks information for the relevant prior snapshot. The client compares its computed chunk hashes against that set and only uploads the chunks the server does not already have.
3. The client invokes the session-relative dynamic_chunk endpoint (variable-size, used for file-level data) or fixed_chunk endpoint (constant-size, used for raw block devices) to upload missing chunks. The server stores each chunk in the datastore's chunk store under its hash and acknowledges receipt.
4. The client registers index files via dynamic_index and fixed_index. These index files map logical offsets in the original disk image to chunk hashes.
5. The client uploads the manifest, which lists every index file in the snapshot, the encryption fingerprint if encryption is on, and a few pieces of metadata.
6. The client calls finish on the session. The server verifies the manifest, checks every chunk referenced by every index actually exists in the chunk store, and atomically renames the temporary snapshot directory into the visible namespace.
7. Garbage collection later cleans up any chunks that no remaining snapshot references.

If the client disconnects between step 1 and step 6, the session times out on the server, the temporary directory gets deleted, and no half-snapshot ever appears in proxmox-backup-manager snapshot list. This is the resumability guarantee from the previous section, made concrete.

The chunk-store deduplication that step 2 leans on has its own write-up if you want to chase it further: how the PBS chunk store deduplicates.

What Wireshark Sees

If you point Wireshark at a running PBS backup, here is what you actually get.

A clean TLS handshake against the PBS port (default 8007). ALPN in the ClientHello advertises h2, http/1.1. The ServerHello picks h2. From there, encrypted application data flows in both directions. Without the TLS keys, that is all you see at the protocol level. Wireshark cannot decode the HTTP/2 frames, let alone the API calls inside them.

Traffic shape is still informative. You can see total bytes uploaded per session, packet timings, and TLS record boundaries. If you know the uncompressed snapshot size, you can infer the deduplication ratio from how many bytes actually went over the wire. Cross-region latency shows up as gaps between request and response on chunk-existence checks, which is the single best diagnostic for "why is my backup slow over the WAN".

If you control the client, you can do better. Skip the packet capture entirely and ask the client to log its own protocol activity. The Rust TLS stack the client uses does not honour SSLKEYLOGFILE in release builds, so trying to feed keys into Wireshark is a dead end for most operators. The client's own debug logging is more informative anyway, because it shows API calls directly instead of HTTP/2 frames you then have to interpret.

proxmox-backup-client backup root.pxar:/etc \
    --repository backup@pbs!nightly@pbs.example.com:store01 \
    --verbose

# For even more detail:
RUST_LOG=trace proxmox-backup-client backup ...

RUST_LOG=trace exposes the same request and response sequence you would otherwise reconstruct from a decrypted packet capture, without needing the TLS keys at all. This is the protocol log you want when a backup hangs and you need to see who is waiting on whom.

Why It Matters for Operations

The protocol is not just trivia. It changes what you can and cannot do operationally.

Performance is round-trip bound. Every chunk-existence check is a round trip. On a 5 ms LAN that is invisible. On a 60 ms intercontinental link with a few hundred thousand chunks per backup, those round trips dominate. Hub placement matters. So does HTTP/2 keepalive. If you want a target close to the data you protect, our edge locations exist for that reason.

You cannot just stick anything in front of PBS. CDN does not work. An HTTP/1.1-only reverse proxy breaks the connection at ALPN, because the client refuses to negotiate anything other than h2. A web application firewall that buffers full request bodies before forwarding will break large chunk uploads. If you need to expose PBS through something, it has to be HTTP/2 end to end, and it has to be transparent.

Debugging starts at the protocol, not the disk. When a backup hangs, the question is almost never "is the disk full". It is "which API call is stuck". --verbose plus the systemd journal on the PBS host tells you within a minute whether the client is waiting on a chunk upload, an index registration, or a finish call. The disk is the third or fourth thing you check.

The round-trip story beats S3-backed tools. restic and borg also do content-defined chunking and deduplication, so the raw bytes on wire in steady state are comparable. The difference is round-trip count and per-chunk protocol overhead. Tools running against an S3 backend pay a HEAD or GET per chunk to decide whether to upload it, plus the latency floor of a blob-store API call. PBS hands the client a known-chunks list up front and runs the rest of the session over a single multiplexed HTTP/2 connection. On a cold S3 backend with hundreds of thousands of chunks, that overhead dominates wall time. If your offsite link has any meaningful round-trip time, this is a real win.

Wrapping Up

The PBS wire protocol is bespoke, opinionated, and tightly coupled to how the rest of the system works. It speaks HTTP/2 over TLS, uses tickets for humans and tokens for machines, and shapes every interaction around making chunk-existence checks cheap. You will not find a stable spec for it, and you should not try to reimplement it. You should understand it well enough to deploy it without breaking it, and well enough to debug it when it sulks.

The next post in this series picks up where the chunk hashes appear: how Proxmox Backup Server actually splits a disk into chunks, why the boundaries are content-defined and not fixed, and what that means for deduplication ratios. Continue on to PBS chunking explained.