Repository navigation

pg_walserver: standalone replication-protocol server (WAL/basebackup serve + push/pull archiving) - #1193

Open

dimitri wants to merge 130 commits into

mainfrom

pg_walsender-server

Open

pg_walserver: standalone replication-protocol server (WAL/basebackup serve + push/pull archiving)#1193
dimitri wants to merge 130 commits into
mainfrom
pg_walsender-server

Commits on Oct 14, 2026
- common: add ifaddr and server-side SCRAM, adapted from PostgreSQL
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  27027bf
- pg_walsender: new standalone replication-protocol server
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  7d68ac8
- pg_walsender: fix copyright headers, dedupe ws_write_file_atomic
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  8810d26
- pg_walsender: port the replication grammar to flex/bison (repl_scanner.l/repl_gram.y), and fscanf-based read_backup_label
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  7823dac
- pg_walsender: rewrite hba.c's line tokenizer to mirror PostgreSQL's next_token()
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  22c8d9b
- pg_walsender: dispatch sub-commands through the project's CommandLine framework
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  74a5e11
- pg_walsender: refresher.c uses the project's PGSQL wrapper + retry policy
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  23314e3
- pg_walsender/scram.c: add missing function header comments
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  3e9da5a
- pg_walsender: remove the monitor-backed HBA keyword, move the FETCH_FILE client to common/
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  6547b80
- pg_walsender: add a comprehensive README.md
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  fe8f471
- docs: add a pg_walsender man page
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  fe56526
- tests: scope pg_walsender_standalone's hand-written HBA rule to loopback, not 0.0.0.0/0
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  c9d2453
- routes.ini: generalize the route-key docs, add a PgBouncer-style "*" wildcard route
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  6897305
- docs: fix pg_walsender.rst's Synopsis to show real --help output, not a fabricated one
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  caa5de4
- docs: design pg_walsender's growth into pg_walserver, a complete standalone archiver
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  0ed7dda
- Rename pg_walsender to pg_walserver
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  1bf07db
- Add setup/fetch-systemid/basebackup sub-commands, rename pg_walserver.ini, real-standby test coverage
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  af1195d
- docs: design routing beyond dbname (SNI, client cert CN) for real physical standbys
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  2610353
- pg_walserver: TLS SNI-based routing for more than one route
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  b24e9c4
- pg_walserver: create-cert sub-command, and CHECK_FILE/ARCHIVE_FILE push side
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  2c71fa8
- pg_walserver: citus_indent fixes for cli_archive.c/cmd_archive_file.c/cmd_check_file.c
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  322fd52
- pg_walserver: embedded, supervised pull capturer (capture = pull)
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  49ec651
- pgaftest: add a per-node 'command' override, test pg_walserver as a container's real PID 1
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  a94d583
- pg_walserver: remove DESIGN-standalone-archiving.md, fully superseded
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  52956a6
- docs: SNI routing needs its own DNS entry per route, an external prerequisite
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  c51783b
- pgaftest: add 'postgres <name>' / 'pg_walserver <name>' cluster-level node sugar
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  e097d0a
- supervisor: genuinely share the MaxR/MaxT restart-backoff ring buffer and PID-1 orphan-reap classification
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  11aa0d6
- pg_walserver: add missing function header comments
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  a12dd18
- pg_walserver: remove dangling DESIGN-standalone-archiving.md reference
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  12af411
- pg_walserver: add restore-wal client sub-command, default capture to pull
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  e2a2b7f
- docs: document restore-wal/archive-wal rename, capture-pull default, pid1 spec
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  dd8fb67
- tests: cover restore-wal and the capture-pull default in pg_walserver specs
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  55856c9
- docs: rewrite pg_walserver.rst in the project's own manual-page register
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  f5084e8
- docs: ground pg_walserver.rst in the production archiving problem it solves
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  cc464fb
- docs: correct archive_library rationale, note persistent-connection mode as planned
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  bfe0c19
- docs: clarify that pg_walserver's routing dbname is not a real database
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  8308542
- pgaftest: auto-generate a usable default archiver-hba.conf for pg_walserver DSL nodes
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  801ac1c
- tests: full standalone archiving+PITR+secondary integration test
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  c3de606
- pg_walserver: archive-wal picks CHECK_FILE-only vs ARCHIVE_FILE-only from the route's own capture setting
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  1d8b699
- tests: update pg_walserver_archive_command.pgaf for the CHECK_FILE/ARCHIVE_FILE split
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  b1478c5
- docs: describe archive-wal's CHECK_FILE-only/ARCHIVE_FILE-only split
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  a362759
- pg_walserver: pidfile + SIGHUP config reload (routes/HBA/capturer)
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  9a1c624
- pg_walserver: drop redundant hand-written help block from ws_root
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  793049f
- pg_walserver: prime the embedded pull capturer before setup --with-basebackup
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  e55bb69
- pg_walserver: serve bootstraps a route's first base backup automatically
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  ae44de8
- pg_walserver: rename --route to --cluster, and archiver-* files to pg_walserver_*
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  5873193
- pg_walserver: add "clientcert=verify-full" HBA option and --ssl-ca-file
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  634d8e5
- docs/tests: use -X stream for real streaming-standby pg_basebackup examples
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  ce653c0
- pg_walserver: dedup WsArchiveTarget/WsRestoreTarget into shared WsWalServerTarget
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  45b7419
- docs: restructure pg_walserver.rst's Access control section
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  9a68020
- pg_walserver: add archive-cleanup (WAL/base-backup retention)
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  c079429
- pg_walserver: add ps, ls, status, and list clusters/backups/wal sub-commands
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  fed9f12
- docs: document ps, ls, status, and list clusters/backups/wal
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  0401490
- tests: add pg_walserver_list_ps_status.pgaf tap spec
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  9929b1b
- pg_walserver: fix cli_root.c sub-command doc count, citus_indent formatting
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  cd58a94
- pg_walserver: rename the embedded pull-capturer subsystem from "capture" to "receivewal"
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  469b7ca
- pg_walserver ps: render as a pstree-style tree instead of a flat table
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  98c7936
- pg_walserver ls: omit config/credential/certificate files by default, add --config
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  8bdeaf3
- pg_walserver: pick pg_basebackup by the upstream's recorded major version
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  e82d015
- pg_walserver archive-cleanup: refuse to delete through a WAL gap
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  7dcddcb
- pg_walserver: relay the embedded receivewal worker's live LSN to ps/status/list clusters
  Dimitri Fontaine
  committed
  03159e1
- pg_walserver basebackup: add --keep-count/--keep-age, composing with archive-cleanup
  Dimitri Fontaine
  committed
  f295d06
- docs: rewrite pg_walserver.rst register, split sub-commands into their own pages
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  a735f45
- docs: fix Manual Pages nav (4 flat top-level entries), rewrite pg_receivewal.rst
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  427cc98
- docs: split pg_walserver into per-command pages like pg_autoctl, rewrite pg_receivewal's hook section
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  39b48e4
- docs: rewrite pg_walserver.rst's dbname-routing-key paragraph for operators
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  338bef1
- docs: rewrite prose that opened with a code span as its grammatical subject
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  0d5b189
- docs: self-review pass — vary repetitive phrasing, drop internal jargon, fix a real example gap
  
  dimitri
  authored and
  Dimitri Fontaine
  committed
  9b61239
- docs: fold in pending fixes from later review rounds (list.rst visible list, pg_receivewal hook correction, create-cert coverage, standalone example split, and smaller trims)
  Dimitri Fontaine
  committed
  96081e0
- docs: reconcile with all 7 landed code changes, real captured output throughout
  Dimitri Fontaine
  committed
  138b074
- pg_walserver: default --path to <pgdata>/<cluster>, persist it to the ini
  Dimitri Fontaine
  committed
  7124b26
- pg_walserver: give the embedded receivewal worker a real replication slot
  Dimitri Fontaine
  committed
  c0c3941
- pg_walserver: fix ARCHIVE_FILE writing received bytes to the client, not disk
  Dimitri Fontaine
  committed
  9966c60
- pg_walserver: fix spurious PGDATA log; redesign 'ls' as a storage summary
  Dimitri Fontaine
  committed
  396cc1a
- docs: genuine captured output, ls/receivewal/archive-cleanup rewrites
  Dimitri Fontaine
  committed
  eab3738
- docs: real captured output for every remaining pg_walserver_*.rst page
  Dimitri Fontaine
  committed
  40a5f84
- pg_walserver: add --ssl-self-signed to setup; postgres:// URIs already worked
  Dimitri Fontaine
  committed
  8d206bb
- pg_walserver: no implicit serve, add stop/help, trim status, fix ERROR-on-first-run
  Dimitri Fontaine
  committed
  96991c5
- fetch_client.c: reuse pgsql.c's connection facility; add missing header comments
  Dimitri Fontaine
  committed
  b0a96ae
- pg_walserver: split "setup" into "setup" (server config) + "cluster" (register/drop/list/set-upstream)
  Dimitri Fontaine
  committed
  c674edc
- pg_walserver: verb-noun CLI (register/drop/set-upstream cluster), --pguri, config-file/--pgdata split
  Dimitri Fontaine
  committed
  9c7901d
- file_utils.c: fix a false-positive "BUG: sformat..." ERROR in set_ps_title()
  Dimitri Fontaine
  committed
  910358d
- pg_walserver: revert cluster CLI to `cluster <verb> <name>`, add CHECK_FILE/ARCHIVE_FILE smart fallback
  Dimitri Fontaine
  committed
  2db50eb
- docs: pg_walserver.rst's "help output, verbatim" was missing the `list` tree
  Dimitri Fontaine
  committed
  5c89bb6
- docs: fix accumulated staleness across pg_walserver reference pages
  Dimitri Fontaine
  committed
  f7e26cf
- pg_walserver: --config rename, table formatting, UPSTREAM pivot, setup HBA/cert auto-provisioning
  Dimitri Fontaine
  committed
  1250718
- pg_walserver ls: replace the boolean --config with --all (-a), free --config for the config-file path flag
  Dimitri Fontaine
  committed
  136710a
- pg_walserver list wal: pivot the default aggregate-stats output, pg_controldata style
  Dimitri Fontaine
  committed
  1a71ca5
- pg_walserver cluster: drop disables instead of removing, add list --disabled and cluster prune
  Dimitri Fontaine
  committed
  9109b35
- pg_walserver: new "cluster enable" command to re-enable a dropped cluster
  Dimitri Fontaine
  committed
  f32ccfc
- docs: split CHECK_FILE and ARCHIVE_FILE into separate wire protocol entries, document CHECK_FILE's columns
  Dimitri Fontaine
  committed
  143891c
- docs: rewrite ARCHIVE_FILE's own entry clearly, end-user style
  Dimitri Fontaine
  committed
  39108a3
- docs: split replication slot commands into separate protocol entries
  Dimitri Fontaine
  committed
  b5e737e
- pg_walserver: track replication slot restart_lsn from real feedback, enforce it in archive-cleanup
  Dimitri Fontaine
  committed
  93aada4
- pg_walserver: fix 4 real gaps found reviewing the disabled-cluster feature
  Dimitri Fontaine
  committed
  26ac939
- pg_walserver: serialize START_REPLICATION per slot with flock(), refuse DROP of an active slot
  Dimitri Fontaine
  committed
  475ecb1
- ipaddr: add fetchLocalIPAddressForRouting(), the UDP counterpart to fetchLocalIPAddress(); use it in pg_walserver's own CIDR auto-discovery
  Dimitri Fontaine
  committed
  954fdf5
- process_supervisor.c: add missing function-level header comments
  Dimitri Fontaine
  committed
  4c1cd50
- pg_walserver: application_name underscore convention, fformat over printf, shared pretty_print_bytes()
  Dimitri Fontaine
  committed
  830e97e
- pg_walserver: colocate each CLI command's getopt/CommandLine with its own run function, matching pg_autoctl's cli_root.c convention
  Dimitri Fontaine
  committed
  b1f57ac
- pg_walserver: move each CommandLine def near top of file, matching pg_autoctl's convention
  Dimitri Fontaine
  committed
  0dfb8c0
- pg_walserver: fix signed-shift UB in framing, add TLS 1.3 ciphersuites, request standby replies
  Dimitri Fontaine
  committed
  5a01e5e
- pg_walserver: finish moving every function body below its file's CommandLine
  Dimitri Fontaine
  committed
  e254906
- pg_walserver: layer archive/restore client, --keep-age parsing, and WAL
  Dimitri Fontaine
  committed
  21d33fd
- pg_walserver: move fetch_client.c/push_client.c out of src/bin/common/ into src/bin/pg_walserver/
  Dimitri Fontaine
  committed
  ea448b4
- Move file CRC32C-over-a-file computation to src/bin/common/file_crc32c.c
  Dimitri Fontaine
  committed
  d0adcc2
- pg_walserver: dedupe cli_ps.c/cli_status.c uptime formatting onto IntervalToString()
  Dimitri Fontaine
  committed
  85773ce
- tar_stream.c: wire the excludeNames parameter through to tar_stream_directory
  Dimitri Fontaine
  committed
  6be5833
- string_utils.c: drop stray ws_/Ws prefixes, match this file's own naming
  Dimitri Fontaine
  committed
  4ba4753
- Move function doc comments from .h to .c, matching project convention
  Dimitri Fontaine
  committed
  d445ce9
- Move remaining .h-pointer function doc comments into their .c files
  Dimitri Fontaine
  committed
  b272d88
- file_crc32c.c/.h: move the missed function doc comment from .h to .c
  Dimitri Fontaine
  committed
  8ae9f5b
- Move remaining silent .h-only function doc comments into their .c files
  Dimitri Fontaine
  committed
  19816e4
- pg_walserver: rename the WsRoute concept to WsCluster throughout the code
  Dimitri Fontaine
  committed
  0359f8c
- docs: rename "route" to "cluster" for the pg_walserver.ini concept
  Dimitri Fontaine
  committed
  6b0ca15
- pg_autoctl: add NODE_KIND_WALSERVER node kind
  Dimitri Fontaine
  committed
  c185c91
- pgaftest: generate walserver nodes through pg_autoctl, support create/launch deferred
  Dimitri Fontaine
  committed
  842cd5f
- pgaftest: add "port <n>" option for pg_walserver DSL nodes, default to 6543
  Dimitri Fontaine
  committed
  2a689ee
- tests: convert pg_walserver_*.pgaf specs to the new walserver node kind
  Dimitri Fontaine
  committed
  3ff914e
- docs: update pgaftest.rst for the new walserver node kind and port/deferred modifiers
  Dimitri Fontaine
  committed
  7bc4812
- pg_walserver archive-cleanup: flatten header comment, move cutoff/label/WAL-continuity helpers into common/, extract backup_list.c
  Dimitri Fontaine
  committed
  634ac21
- pg_walserver archive-cleanup: split the ~400-line run function into named phase helpers
  Dimitri Fontaine
  committed
  f67c7fb
- pg_walserver cli_*.c: flatten dash-divider box comments to plain prose
  Dimitri Fontaine
  committed
  e9a34fd
- pg_walserver cli_*.c: rename 6 cross-file APIs off the cli_ prefix onto ws_
  Dimitri Fontaine
  committed
  6ee01dc
- pg_walserver: split cli_serve_run and ws_setup_execute into named phase helpers
  Dimitri Fontaine
  committed
  a9f4037
- pg_walserver/common: de-duplicate format_elapsed into common/system_utils.c
  Dimitri Fontaine
  committed
  9ffcdf8
- pg_walserver: close out cli_list.c/cli_ls.c cleanup gaps from the broader CLI sweep
  Dimitri Fontaine
  committed
  10d1393
- pg_walserver/pg_autoctl: stop bypassing file_utils.h's own facilities
  Dimitri Fontaine
  committed
  e0472b1
- pg_autoctl: build file writes in memory (PQExpBuffer) and write once
  Dimitri Fontaine
  committed
  fe20411
- pg_autoctl: pghba.c's pg_hba.conf rewrites use write_file_atomic
  Dimitri Fontaine
  committed
  fb117fd

Commits

Commits on Oct 14, 2026