[happy-v587]

Which issue or RFC does this PR close?

Closes #1985.

Implements the optional deployment described in RFC 0046.

Rationale for this change

Operators currently have to assemble collection, storage, dashboards and alert diagnostics themselves. Provide an opt-in reference stack and external-platform connections so they can inspect service health, processing progress and dependencies through correlated metrics, logs and traces.

What changes are included in this PR?

• Package the Docker Compose stack, pinned component configuration, persistent storage, authentication, retention, dashboards, alert rules and runbooks. Include Prometheus, Grafana, Loki, Tempo, OpenTelemetry Collector and Phoenix. Declare the dependencies used by Server process metrics in the Server installation role. Use the same bounded trace/log profile for the reference demo and managed deployment; pin Phoenix to its verified multi-platform image digest and permit bounded localized dashboard queries through the gateway.
• Add powercontext observability up|status|check|down|connect and server run --observability, with existing-server, container, split API/background and native personal-service configuration. Profile opt-in rejects signal-specific trace/log exporter overrides from files and process environments so destinations, authentication and timeouts cannot bypass the selected stack.
• Add bounded Provider HTTP/operation observations and Worker reports; observe outgoing REST status codes, final retrievals, exact Artifact reads, processing state and Bootstrap delivery metadata. Preserve Worker reaping, cancellation, failure diagnostics and managed file logging. Observe the wire delegate inside inference privacy protection: rejected inputs create no HTTP exchange, and withheld responses retain their actual HTTP status.
• Connect Scope/service/Provider dashboards to log and trace evidence, with sampled exemplars where available. Retain the operational field allowlist and exclude content, credentials and execution identities from metric labels.
• Regenerate HTTP/Desktop contracts, record local Desktop protocol qualification, update both documentation locales and add regression/acceptance coverage. Receipt statistics read existing records; there is no database schema or persisted-format migration. Keep macOS image-busy recovery bounded and retain the selected build diagnostics.

Server /metrics -> Prometheus -> Grafana dashboards and alerts
Server/Worker logs and traces -> Collector -> Loki / Tempo / Phoenix
Grafana diagnostics -> related logs and traces when evidence is available

Are there any user-facing changes?

• Users can explicitly install and connect the monitoring stack, or connect an existing OTLP/Prometheus platform. Normal SDK use does not configure logging or require a monitoring backend.
• Grafana exposes service, Scope, processing and Provider diagnostics with drill-down links and documented missing/stale-data semantics. Active exercise probes are explicit.
• Optional statistics fields and monitoring configuration are additive. There are no intentionally breaking public API or stored-data changes. The affected runtime/Server telemetry boundaries make Worker cleanup, cancellation, privacy and export failure isolation the main regression risks.

How was this change tested?

• The real HTTP/SQLite ZCode chain exercises a delayed initial context response beyond the Hook's one-second request budget. The initial binding probe accepts only the documented prepare-timeout diagnostic and still requires the exact Scope binding and a persisted Source; all other Hook calls retain strict diagnostics, durable Stop pause/no-replay checks and factual recall in a new session. This reproduced the CI assertion on Python 3.12 with Node 22.19 before the repair; all four service-chain scenarios now pass. A Python 3.11 selection passed 73 cases, with the unconfigured OceanBase receipt and Windows-only installation cases skipped. The Node 22.19 plugin suite passed 61 cases; its real ZCode CLI host case was skipped because no built CLI bundle was supplied. make check passed, including 61 integration-manifest cases.
• The macOS retry regression executes the actual workflow shell with controlled build results. It reproduced the missing eject-busy recovery before the fix (two failures), then passed all five cases: create/eject busy, terminal compilation error, unrelated resource-busy output and three-attempt exhaustion. After rebase onto master 2f15c8d6, 216 focused CI/runtime/profile cases and make check passed, including 61 integration-manifest cases. The six public log-exporter override regressions and the complete OTLP selection (81 cases, no skips) also passed in the profile repair.
• An isolated real Docker reference stack passed scrape/export authentication, datasource/dashboard provisioning, a sampled request queried from Tempo, its matching trace_id log queried from Loki, and a real metric/exemplar stored in Prometheus. Only test ports changed in temporary demo copies. The first real query exposed HTTP 414 for a localized dashboard expression; the bounded 32 KiB gateway request-line setting fixes it. The pinned Phoenix OCI index was fetched from Docker Registry, its SHA-256 verified, both Linux amd64/arm64 manifests checked, and the exact reference pulled and started.
• Reproduced the lean Server startup failure with an installed wheel in a clean Python 3.11 environment containing only server,cli. After declaring psutil in the Server extra, all five isolated installation roles passed on Python 3.11 and 3.14. OpenDAL passed isolated lint, format, types, CLI and eight tests. macOS arm64 managed-candidate production/consumption passed 56 cases with two Windows-only skips, including repeatable archives, relocation, crypto, independent Ops and a real bundled Server. The built wheel passed four SQLite Desktop protocol modes without the optional Code parser.
• Main run 41644120295 at 6aa1e1d3 passed 86 checks, including the Python 3.11–3.14 unit/e2e matrix, all four SeekDB shards, isolated installations, worker-observability, native managed candidates, Windows/Linux Desktop validation and 11 upgrade configurations. The macOS Intel application compiled and its tests passed, but DMG detach exhausted the upstream resource-busy retries; the workflow now recognizes that specific failure for bounded recovery and retains hidden-directory build diagnostics. The final OceanBase vector+Code upgrade case was cancelled after the updated head was pushed. Main run 41691599801 at b37639cb passed seven checks, including Python 3.11 unit/e2e and Python 3.12 unit tests, before the Python 3.12 e2e acceptance rejected the initial prepare-timeout diagnostic and stopped the remaining matrix. The acceptance now exercises that documented fallback explicitly. Current head f71fa2cd is under verification in Main run 41703722055; actual native packaging and full CI success remain pending.
• API contract checks passed all 62 cases; Desktop generation, type checking, native probe and bundled qualification session checks passed. Historical qualification sqlite-observability-da44b918-v1 retains its exact source, contract and wheel identities, with all 719 packaged runtime files verified against the recorded source. Disposable CI smokes do not replace that qualification.

The fresh Docker execution above covers the host demo and storage/correlation path. Full backend-outage fault injection, the split OceanBase topology and external production services were not rerun in this repair. Local reception and fixture checks do not establish those deployment results.

AI usage statement

OpenAI Codex assisted with implementation, conflict resolution, tests, documentation and this PR description. The validation above distinguishes local execution from GitHub Actions results.

[hidb4ai]

Please address the managed logging configuration conflict below before merging.