Reclaim fully retired idle TLog queues

[mengxu-oai]

Problem

A restored shared TLog queue can keep its high-water file size after every retained log generation retires. An idle queue receives no later payload commit to advance the restored committed boundary, persist retirement pops, or rotate its files.

Change

• Commit recovery repair writes and establish the restored queue's committed boundary.
• Persist retirement-only queue pops after retirement metadata is durable. Pop trailing padding only after every generation retires.
• Reclaim an oversized, fully popped two-file queue with a durable empty marker. Keep one durable marker while replacing the files, and serialize later writes with replacement. The disk format is unchanged.
• Reuse GcGenerations with Cycle for regression coverage. Retain old log roles during the cleanup phase so automatic role removal cannot hide a leaked queue. Require fresh metrics from the restored, idle target queue.
• Check idle shared-queue disk usage in the general end-of-simulation consistency check. Use each queue's actual allocation limits, including buggified values.
• Restart the target machine during recovery and retirement. Observe the simulated filesystem and crash during either atomic queue-file replacement, then require recovery, disk cleanup, and final consistency checks.

The dedicated cleanup workload and all test-specific TLog callbacks are removed.

Validation

• Controlled unfixed comparison: the same revised test and diagnostics fail the idle-queue disk check. The retired queue retains 8,613,888 bytes, above its 2,170,880-byte allocation limit. The first error is TLogQueueCleanupPending.
• Fixed end-to-end simulations: 3/3 passed, including final consistency checks and no unexpected severity-error events. All reached an idle queue size of 8,192 bytes.
  • Normal cleanup: seed 301, no extra cleanup restarts.
  • Machine-crash coverage: seeds 1 and 2 with three cleanup restart iterations. Trace evidence confirms actual machine reboots while the second and first atomic replacement files, respectively, are pending; both recover and pass final consistency checks.
• Native unit suites: 147 passed, 0 failed — core 38, key-value store 27, TLog 15, log system 26, and ClusterController 41.
• Clang-format 19 and whitespace checks passed.
• The revised test still needs bulk correctness validation. Both comparison and fixed packages are prepared; the requested 100,000-run full correctness campaign has not started.

Run the checked-in regression with simulation tests enabled:

build/bin/fdbserver -r simulation -f tests/slow/GcGenerationsQueueCleanup.toml --seed 1 --buggify off