Skip to content

Refuse a bulkload restore of a partitioned-log backup - #14311

Closed
saintstack wants to merge 1 commit into
apple:mainfrom
saintstack:dev/stack/bulk-plog-refusal
Closed

saintstack wants to merge 1 commit into
apple:mainfrom
saintstack:dev/stack/bulk-plog-refusal

Conversation

@saintstack

Copy link
Copy Markdown
Contributor

Problem

StartFullRestoreTaskFunc reads mutationLogType and then takes the !useRangeFileRestore branch without consulting it, so RestoreDispatchPartitionedTaskFunc is unreachable for a bulkload restore and the plain RestoreDispatchTaskFunc is handed partitioned log files.

Nothing refused the combination: mutationLogType appears nowhere in submitBackup, none of the existing bulkload refusals mentioned it, and the CLI accepts fdbbackup start --mode bulkdump --mutation-log-type partitioned-log followed by fdbrestore start --mode bulkload.

No data is corrupted. decodeMutationLogFileBlock rejects a plog on its first block, since the block format version is PARTITIONED_MLOG_VERSION (4110) rather than BACKUP_AGENT_MLOG_VERSION (2001).

But the restore never fails either. A restore task that throws is logged and retried, not aborted: TaskBucket records lastError and never finish()es the task, so it is re-dispatched on every timeout expiry with no retry cap and no transition to ERestoreState::ABORTED. The observable result is a restore sitting at zero log bytes applied, repeating restore_unsupported_file_version, rather than one that reports the layout cannot be replayed.

Fix

Refuse it at submit time through bulkLoadRestoreRefusal(), which already receives mutationLogType by way of submitRestore — so no new plumbing is required.

That value comes from desc.mutationLogType, which means the refusal keys off the layout describeBackup actually found in the container, not off a flag the caller happened to pass. That matters here: getRestoreSet selects the partitioned layout from container contents (if (!plogs.empty())), so the hazard does not require the user to have requested anything.

Known gap

This does not cover a container that holds partitioned logs while desc.mutationLogType comes back DEFAULT, which can happen because describeBackup only sets the property when the plogs/ listing for its scan window is non-empty. That detection gap is a separate problem and is not addressed here.

A belt-and-braces alternative would be a consumer-side check in the classic reader, for which the data is already plumbed — RestoreConfig::RestoreFile carries tagId/totalTags, and LogFile::isPartitionedLog() exists but currently has zero callers. I have left that out to keep this change to the submit-time refusal; happy to add it if preferred.

Testing

Covered by the existing TEST_CASE("/backup/restore/bulkLoadRestoreRefusal"), extended with three cases: a partitioned log is refused for bulkload, a classic log is not, and rangefile mode is unaffected.

Not vacuous: with the new check replaced by if (false), the test fails (0 tests passed; 1 tests failed).

fdbclient_test --simulation -f /backup/ → 32 passed, 1 failed. The failure is /backup/time (parseTime(formatTime(ts)) == ts), which fails identically on unmodified main on this machine and is unrelated.

No existing test combines partitioned logs with bulkload, so nothing regresses.

StartFullRestoreTaskFunc reads mutationLogType and then takes the
!useRangeFileRestore branch without consulting it, so the partitioned
dispatcher is unreachable for a bulkload restore and RestoreDispatchTask
is handed partitioned log files. Nothing refused the combination:
mutationLogType appears nowhere in submitBackup, none of the existing
bulkload refusals mentioned it, and the CLI accepts --mode bulkdump with
--mutation-log-type partitioned-log followed by --mode bulkload.

The data is never corrupted -- decodeMutationLogFileBlock rejects a plog
on its first block, because the block format version is not
BACKUP_AGENT_MLOG_VERSION. But a restore task that throws is logged and
retried rather than aborted: TaskBucket records lastError and never
finishes the task, so it is re-dispatched on every timeout with no retry
cap and no transition to ERestoreState::ABORTED. The restore sits at zero
log bytes applied, repeating restore_unsupported_file_version, instead of
reporting that the layout cannot be replayed.

Refuse it at submit time through bulkLoadRestoreRefusal, which already
receives mutationLogType by way of submitRestore. The value comes from
desc.mutationLogType, so this keys off the layout describeBackup found in
the container rather than off a flag the caller passed.

Note this does not cover a container that holds partitioned logs while
desc.mutationLogType comes back DEFAULT, which can happen because
describeBackup only sets it when the plogs listing for its scan window is
non-empty. That detection gap is a separate problem.

Covered by the existing bulkLoadRestoreRefusal unit test: a partitioned
log is refused for bulkload, a classic log is not, and rangefile mode is
unaffected. With the check disabled the test fails.
@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-ide on Linux RHEL 9

  • Commit ID: d56be9b
  • Duration 0:20:33
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: d56be9b
  • Duration 0:23:45
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: d56be9b
  • Duration 0:24:55
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: d56be9b
  • Duration 0:25:25
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux RHEL 9

  • Commit ID: d56be9b
  • Duration 0:35:53
  • Result: ❌ FAILED
  • Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS 14.x

  • Commit ID: d56be9b
  • Duration 0:44:00
  • Result: ❌ FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@saintstack saintstack closed this Oct 17, 2026
@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS 14.x

  • Commit ID: d56be9b
  • Duration 1:20:26
  • Result: ❌ FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants