[mtenpow]

Summary

The gcp_pubsub source creates its batch finalizer and acknowledgement channel per streaming
pull and drops both when the stream ends. Pub/Sub ends streaming pulls routinely (UNAVAILABLE
while it rebalances), the source restarts them when its token is refreshed, and any error ends
them. Every batch still in the pipeline at that moment loses its acknowledgement, and its
messages are redelivered once their ack deadline expires. The source also opens streams with
no flow control, so behind a slow sink it leases far more messages than it can deliver within
the ack deadline.

In a 1.2M-message run behind a 1,000 events/s sink (ack deadline 600 s), this produced 34,176
duplicate deliveries, all expiring together 600 s after a stream restart.

This PR:

• keeps the finalizer, pending-batch count and acknowledgement queue per stream task instead
  of per stream. Pub/Sub accepts ack ids on any stream of the subscription, so acks for
  batches finalized after their stream ended, and updates the ended stream had not sent, go
  out on the next stream. A stream counter kept under the queue's lock ends the previous
  request stream when a new one takes over the queue, so an ended stream cannot take updates
  meant for its successor. The server answers a new streaming pull only once it has messages
  to deliver, so the startup wait now forwards finalized batches too, and waits for pending
  batches before shutting down, as the main loop already did;
• polls the finalizer stream only while batches are pending, since it ends after shutdown and
  an ended stream would always win the biased select;
• releases the leases (modify_deadline_seconds = 0) of messages the source failed to emit;
• adds max_outstanding_messages and max_outstanding_bytes, sent in the first
  StreamingPullRequest. Both default to 0 (no limit), so existing configurations are unchanged.

The vendored proto has only the StreamingPull RPC, so carrying acks to the next stream is
smaller than adding a unary Acknowledge call.

The change was developed and qualified against v0.54.0, where we run it, and ported to
master for this PR. The port kept the mechanism unchanged and adapted it to the pedantic
clippy lints master now enables.

References

None (no existing issue).

Vector configuration

sources:
  pubsub:
    type: gcp_pubsub
    project: my-project
    subscription: my-subscription
    ack_deadline_secs: 600
    max_concurrency: 2
    max_outstanding_messages: 20000
    max_outstanding_bytes: 67108864

How did you test this PR?

New unit tests run against an in-process tonic implementation of the vendored Subscriber
service, whose first streaming pull delivers one message and then ends with UNAVAILABLE:

• acknowledges_batches_finalized_after_their_stream_ended: the event is finalized only after
  the source opened its next stream, and the ack must reach the server;
• shuts_down_after_its_pending_batches_are_finalized: shutdown begins while the batch is
  pending and the next stream is starting, and the source must wait for the batch, then finish;
• requests_the_configured_flow_control and flow_control_is_unlimited_by_default.

On master (7fd2b32), x86_64 Linux, Rust 1.96.1:

• with only the test commit applied, both regression tests fail (timed out waiting for an acknowledgement; the source shut down with a batch pending);
• with the whole PR, cargo test --no-default-features --features sources-gcp_pubsub --lib -- sources::gcp_pubsub:: passes all 7 tests, and 10 more consecutive runs pass;
• cargo clippy --no-default-features --features sources-gcp_pubsub --lib --tests -- -D warnings
  and cargo fmt --check are clean;
• vdev check changelog-fragments accepts both fragments;
• vector generate-schema and vdev build component-docs (cue 0.17.1) produce the committed
  max_outstanding_messages and max_outstanding_bytes entries in
  website/cue/reference/components/sources/generated/gcp_pubsub.cue. That run used a
  single-component build, so I did not run make check-generated-docs over the full schema.

On v0.54.0 the same tests showed the same red/green result, and the fake-server tests passed 20
more consecutive runs. The Pub/Sub emulator integration tests were not run (they are ignored
upstream, #24133).

Change type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Does this PR include user facing changes?

• Yes. Changelog fragments are in changelog.d/.
• No. A maintainer will apply the no-changelog label to this PR.

[github-actions]

Thank you for your contribution! Before we can merge this PR, please sign our Contributor License Agreement.

To sign, copy and post the phrase below as a new comment on this PR.

Note: If the bot says your username was not found, the email used in your git commit may not be linked to your GitHub account. Fix this at github.com/settings/emails, then comment recheck to retry.

---

I have read the CLA Document and I hereby sign the CLA

---

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}