[saintstack]

fdbrestore status is text-only. Automation has to regex-parse it, and it emits two different formats depending on the snapshot engine — blocks and files for a rangefile restore, task counters for a bulkload one.

fdb-kubernetes-tests carries a standing TODO for exactly this and parses both shapes by hand today (tests/fixtures/backup_restore_fixture.go:1152-1157).

fdbbackup status --json and fdbbackup describe --json already exist; restore was the gap.

Approach

RestoreConfig::getProgress_impl and getFullStatus_impl each performed their own reads and built text by format() concatenation, with the latter calling the former. Rather than add a second, independent JSON reader, the reads move into one RestoreConfig::StatusSnapshot and all three outputs become pure renderers over it, sharing the snapshot-phase decision via statusPhases().

This shape is deliberate. The last JSON defect in this area — fdbbackup status --json serializing an empty Errors array (#14124) — was caused precisely by a JSON path written separately from the text path it was meant to mirror, and drifting. One source of data makes that class of bug structurally harder.

getProgress() and getFullStatus() keep their signatures and their output is byte-identical, so no call site needed changing.

One consequence worth recording, because "callers are unaffected" would be too strong: the consolidated read set is a superset of what getProgress() alone used to read, and getProgress()'s only caller is waitRestore(), which commits its transaction — it registers a watch on stateEnum and commits a few lines later. So six config reads that the progress renderer never consumes now enter that transaction's read-conflict set: addPrefix, removePrefix, sourceContainerURL, restoreVersion, bulkLoadComplete, and getRestoreRangesOrDefault(). The last is a paginated range read over the restoreRangeSet subspace rather than a point read, so it contributes a range conflict rather than a single key. (stateEnum is not an addition — getProgress() already read it via stateText().)

The practical cost is nevertheless very low. bulkLoadComplete is the only one plausibly written mid-restore, it flips once, the read-version-to-commit window is milliseconds, and that loop already reads stateEnum — the very key that changes to end the wait — so it is conflict-prone by construction and already recovers through tr->onError(). Expect well under one extra retry per restore, and only on fdbrestore wait and start --waitfordone, the paths that pass Verbose::True.

Output

Only the counters that apply to the engine in use are emitted. Zero-filling the inapplicable set would let a consumer read "not applicable" as "no progress" — note the Tasks counters below are legitimately all-zero early in a bulkload restore, so an emitted Blocks: 0/0 alongside them would be genuinely ambiguous.

An empty tag reports every restore, so the result is one object with a Restores array rather than a document per tag. That also means a single document can contain both engine shapes, which the concatenated text format cannot express cleanly.

Captured live from the s3_backup_bulkdump_bulkload ctest, which runs a bulkload restore alongside a rangefile validation restore:

{
    "SchemaVersion": "1.0.0",
    "Restores": [
        {
            "Tag": "test_backup_bulkdump",
            "UID": "cd179a2e3c8cdfaf4492e9df4ba99d99",
            "State": "running",
            "SnapshotMethod": "bulkload",
            "URL": "blobstore://127.0.0.1:8081/ctests/94135/test_bulkdump_bulkload?bucket=test-bucket&region=us-east-1&secure_connection=0",
            "Ranges": [ { "Begin": "", "End": "\\xff" } ],
            "AddPrefix": "",
            "RemovePrefix": "",
            "RestoreVersion": 21723915,
            "CurrentVersion": 1853165,
            "FirstConsistentVersion": 1853165,
            "BytesWritten": 0,
            "Tasks": { "Submitted": 0, "Triggered": 0, "Running": 0, "Total": 0 },
            "SnapshotPhase": "in_progress",
            "MutationLogPhase": "not_started"
        },
        {
            "Tag": "test_backup_bulkdump_validate",
            "UID": "1bc3c8f007212f64f74b0ab9377ea53a",
            "State": "completed",
            "SnapshotMethod": "rangefile",
            "URL": "blobstore://127.0.0.1:8081/ctests/94135/test_bulkdump_bulkload?bucket=test-bucket&region=us-east-1&secure_connection=0",
            "Ranges": [ { "Begin": "", "End": "\\xff" } ],
            "AddPrefix": "\\xff\\x02/rlog/",
            "RemovePrefix": "",
            "RestoreVersion": 21723915,
            "CurrentVersion": 21723915,
            "FirstConsistentVersion": 1853165,
            "BytesWritten": 330,
            "Blocks": { "Finished": 1, "Total": 1, "InProgress": 0 },
            "FileCount": 2,
            "ApplyVersionLag": 0,
            "SnapshotPhase": "complete",
            "MutationLogPhase": "complete"
        }
    ]
}

And a single-tag rangefile restore, before and after completion, from a file:// container:

{"SchemaVersion":"1.0.0","Restores":[{"Tag":"testtag","UID":"629eef09732e61b06863dabeeb7d5faf",
"State":"queued","SnapshotMethod":"rangefile","URL":"file:///...","Ranges":[{"Begin":"","End":"\\xff"}],
"AddPrefix":"","RemovePrefix":"","RestoreVersion":24125773,"CurrentVersion":-1,
"FirstConsistentVersion":0,"BytesWritten":0,"Blocks":{"Finished":0,"Total":0,"InProgress":0},
"FileCount":0,"ApplyVersionLag":0}]}

{"SchemaVersion":"1.0.0","Restores":[{"Tag":"testtag","UID":"629eef09732e61b06863dabeeb7d5faf",
"State":"completed",...,"BytesWritten":222,"Blocks":{"Finished":1,"Total":1,"InProgress":0},
"FileCount":2,"ApplyVersionLag":0,"SnapshotPhase":"complete","MutationLogPhase":"complete"}]}

Notes on the shape:

• SnapshotPhase/MutationLogPhase are emitted only for RUNNING and COMPLETED, matching the text path — hence absent while queued.
• CurrentVersion: -1 while queued is pre-existing getCurrentVersion() behaviour surfaced as-is, not introduced here. Consumers should read it as "not yet known".
• Keys are PascalCase with a SchemaVersion, following fdbbackup status --json and describe --json. (fdbbackup query uses snake_case with no schema version; that inconsistency is pre-existing and not propagated here.)

Testing

Both in-tree backup ctests pass on this change, re-run after rebasing onto current main (which upgraded Boost to 1.89.0), and built with WITH_ROCKSDB=ON so the fixture's hardcoded ssd-rocksdb-v1 engine is available:

Test	Result
dir_backup_tests (rangefile, file://)	exit=0, PASSED test_dir_backup_and_restore
s3_backup_bulkdump_bulkload_tests (bulkload, MockS3)	exit=0, PASSED test_bulkdump_bulkload, plus Audit validation PASSED - BulkLoad produces identical results to traditional restore
BackupTests (option parsing over the real CSimpleOpt tables)	21/21, exit=0

These two ctests are also the regression gate for the text output, since both grep it (backup_tests_common.sh matches State:.*completed and Phase:). Text was verified byte-identical by inspection as well: the two original functions computed the bulkload flag with opposite polarity (!present \|\| value in getProgress_impl vs present && !value in getFullStatus_impl); those are exact complements, so collapsing them into one useRangeFile field is behaviour-preserving at every use site.

Additionally verified by hand against a local loopback cluster: full backup → clear → restore → data verified, with --json polled mid-restore and after completion (the single-tag output above), and both documents confirmed parseable.

Incidental fixes

• getProgress_impl computed an errstr from lastError and never used it, so fdbrestore status has never displayed the last error despite reading it. The dead local is gone and the value is now surfaced as LastError: {Message, SecondsAgo} in the JSON. This makes JSON strictly more informative than text here; aligning the text output is deliberately left out of scope to keep it byte-identical.
• getFullStatus previously read useRangeFileRestore twice, once directly and once via getProgress. Now read once.
• stateText(tr) is map(stateEnum().getD(tr), restoreStateText), so reading it alongside stateEnum() fetched the same key twice in one transaction. The text is now derived from the enum: identical value within a single snapshot, and one fewer read on all three status paths.

Not in scope

• fdbdr status still has no JSON at all — DatabaseBackupAgent has no getStatusJSON counterpart.
• fdbbackup error paths are still plain text on stderr regardless of --json, so a --json consumer cannot distinguish "empty result" from "failed".
• fdbbackup list, tags and dump remain text-only.

[akankshamahajan15]

I think we can remove "instead of text" as it is evident from the json word itself

[saintstack]

Done (both places)

[akankshamahajan15]

Curious about the wording "so the two cannot disagree.", why would they disagree? And is this statement needed anymore?

[saintstack]

Dropped. Not something needed in code (refactor rationale).

[foundationdb-ci]

Result of foundationdb-pr-clang-ide on Linux RHEL 9

• Commit ID: 19aa342
• Duration 0:42:33
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-clang-arm on Linux RHEL 9

• Commit ID: 19aa342
• Duration 0:48:30
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr on Linux RHEL 9

• Commit ID: 19aa342
• Duration 1:04:23
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-clang on Linux RHEL 9

• Commit ID: 19aa342
• Duration 1:07:32
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

• Commit ID: 19aa342
• Duration 1:49:20
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)
• Cluster Test Logs zip file of the test logs (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-macos on macOS 14.x

• Commit ID: 19aa342
• Duration 2:02:53
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-macos-m1 on macOS 14.x

• Commit ID: 19aa342
• Duration 1:19:25
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)