Skip to content

refactor!: remove workspace fan-out - #89

Merged
horakihor merged 4 commits into
mainfrom
remove-workspace-fanout
Nov 9, 2026
Merged

horakihor merged 4 commits into
mainfrom
remove-workspace-fanout

Conversation

@horakihor

@horakihor horakihor commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This removes workspace fan-out (TFBUDDY_WORKSPACE_FANOUT_ENABLED, added in #62). With fan-out on, a hook event published one message per workspace to the TFBUDDY_WORKSPACE_TRIGGERS stream, and a separate workspace worker ran each one. Fan-out was added so that a merge request touching many workspaces would not outlive the hook message's AckWait and get planned twice.

#83 solved that problem on the inline path. The hook queues are now pull consumers that keep a message in progress while it runs, and the WORKSPACE_DISPATCHES claim skips workspaces that an earlier delivery already dispatched. Since then, fan-out has only been a second dispatch path that nothing exercises:

  • Zapier's deployment has run with TFBUDDY_WORKSPACE_FANOUT_ENABLED=false since v0.6.0 (September 2026). Fan-out ran in production twice in May 2026, on v0.5.0 and v0.5.1, and was rolled back both times. Nothing has been published to TFBUDDY_WORKSPACE_TRIGGERS since 2026-05-22.
  • On v0.8.5, a single replica handles a 99-workspace merge request inline.

What is removed:

  • pkg/tfc_trigger/workspace_stream.go (the stream, its message type and its jsqueue pull queue) and workspace_worker.go, with their tests.
  • The enqueue branch in TriggerTFCEvents. dispatchWorkspaces now calls triggerRunForWorkspace directly, with the same log line, error text and commit status as the old inline dispatcher.
  • SetWorkspaceStream and the WorkspacePublisher interface, plus the field and constructor argument that carried them through the GitLab and GitHub hook handlers. The Trigger mock is regenerated.
  • The workspace-fanout-enabled and workspace-jetstream-replicas settings.
  • The fan-out sections in the README and docs/. docs/usage.md is regenerated. The chart never set either setting, so it has nothing to remove.

DeliveryID stays, because the dispatch claim and the auto-merge apply generation use it. Only its comments changed.

Behaviour change for other deployments

Fan-out was on by default, and the README and docs presented it as the normal path. A deployment that never set TFBUDDY_WORKSPACE_FANOUT_ENABLED has been running fan-out, and after this change it plans every workspace inline in the hook worker. That is the same behaviour as TFBUDDY_WORKSPACE_FANOUT_ENABLED=false today.

  • The TFBUDDY_WORKSPACE_FANOUT_ENABLED and TFBUDDY_WORKSPACE_JETSTREAM_REPLICAS environment variables are now ignored.
  • The --workspace-fanout-enabled and --workspace-jetstream-replicas flags no longer exist, so a command line that still passes them fails to start.
  • A deployment that had fan-out on loses any workspace messages still in the stream when the last old replica stops. The stream drops them after an hour anyway, and tfc plan re-runs them.

The title is marked breaking (refactor!:), so semantic-release cuts v1.0.0 as the next release.

Merge note: the commit on this branch still says refactor:. Squash-merge, and set the squash commit title to refactor!: remove workspace fan-out before confirming. Otherwise semantic-release does not see the breaking change.

After deploying

TFBuddy no longer creates or reads TFBUDDY_WORKSPACE_TRIGGERS. Once no older release is running, the stream and its leftover push consumer (tfbuddy_workspace_trigger_worker) can be deleted from NATS:

nats stream rm TFBUDDY_WORKSPACE_TRIGGERS

Deleting the stream also deletes the consumer. This PR does not delete anything itself.

Testing

  • go vet ./...: clean.
  • go test -race ./...: all 17 packages pass.
  • go generate ./internal/config/ regenerates docs/usage.md, and mockgen regenerates pkg/mocks/mock_tfc_trigger.go. The only change in the mock is the removed method.

🤖 Generated with Claude Code

Workspace fan-out (TFBUDDY_WORKSPACE_FANOUT_ENABLED, added in #62)
published one message per workspace to the TFBUDDY_WORKSPACE_TRIGGERS
stream and ran each one in a separate workspace worker. It was meant to
stop a slow merge request from outliving the hook message's AckWait.
Since #83 the hook queues are pull consumers that keep a message in
progress while it runs, and the WORKSPACE_DISPATCHES claim skips
workspaces an earlier delivery already dispatched. The inline path
covers the problem fan-out was added for.

Zapier's deployment has run with fan-out disabled since v0.6.0, after
two rollbacks in May 2026. Keeping it means a second dispatch path
that nothing in production exercises.

Remove the workspace stream and worker, the TriggerTFCEvents enqueue
path, the SetWorkspaceStream plumbing through the GitLab and GitHub
hook handlers, and the workspace-fanout-enabled and
workspace-jetstream-replicas settings. Every workspace now runs the way
it did with fan-out disabled. Docs no longer describe fan-out, and say
how to delete the stream that older releases left behind.

Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@horakihor horakihor changed the title refactor: remove workspace fan-out refactor!: remove workspace fan-out Nov 6, 2026
@horakihor
horakihor requested a balanced review from Copilot November 8, 2026 12:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the change to inline-only workspace dispatch.

No open issues 🎉

1 file not reviewed
  • pkg/mocks/mock_tfc_trigger.go: Generated file
What changed in this PR

This PR removes workspace fan-out and makes inline dispatch the only execution path. It relies on the hook-consumer keep-alives and dispatch claims introduced in #83 to handle long-running events and prevent duplicate workspace runs.

Changes:

  • Removes the workspace queue, worker, associated tests, and hook-handler wiring.
  • Dispatches all touched workspaces from one checkout while preserving existing inline error handling and commit statuses.
  • Removes fan-out settings and updates documentation, including cleanup guidance for the retired stream. Existing command lines using the removed flags must be updated.
File Description
README.md Describes inline workspace dispatch and hook keep-alives.
pkg/​vcs/​github/​hooks/​stream_worker.go Removes workspace-stream injection into GitHub triggers.
pkg/​vcs/​github/​hooks/​stream_types.go Clarifies the retained delivery-ID comment.
pkg/​vcs/​github/​hooks/​github_hooks_handler.go Removes the workspace publisher field and constructor argument.
pkg/​tfc_trigger/​workspace_worker.go Deletes the dedicated workspace worker.
pkg/​tfc_trigger/​workspace_worker_test.go Deletes workspace-worker tests.
pkg/​tfc_trigger/​workspace_stream.go Deletes the workspace stream and message implementation.
pkg/​tfc_trigger/​workspace_stream_test.go Deletes stream and fan-out tests.
pkg/​tfc_trigger/​tfc_trigger.go Replaces selectable dispatch paths with direct inline execution.
pkg/​tfc_trigger/​interfaces.go Removes SetWorkspaceStream from the trigger interface.
pkg/​runstream/​workspace_dispatch.go Updates the dispatch-claim TTL comment.
pkg/​mocks/​mock_tfc_trigger.go Removes the obsolete mock method.
pkg/​hooks/​server.go Removes workspace-stream startup, worker setup, and health check.
pkg/​gitlab_hooks/​stream_msgs.go Clarifies delivery-ID comments without changing behavior.
pkg/​gitlab_hooks/​mr_actions.go Removes workspace-stream injection into MR triggers.
pkg/​gitlab_hooks/​hook_worker.go Removes the worker’s workspace publisher dependency.
pkg/​gitlab_hooks/​handler.go Removes the publisher field and constructor argument.
pkg/​gitlab_hooks/​comment_actions.go Removes workspace-stream injection into comment triggers.
internal/​config/​config.go Removes fan-out settings and narrows dedup-window documentation.
docs/​usage.md Updates generated configuration documentation.
docs/​contributing.md Updates the explanation for race-enabled testing.
docs/​architecture.md Documents inline processing and retired-stream cleanup.

Review effort: Balanced

@horakihor
horakihor enabled auto-merge (squash) November 9, 2026 09:49
@horakihor
horakihor disabled auto-merge November 9, 2026 10:15
The DCO app did not report on the merge commit from "Update branch", so the required DCO check never arrived.

Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@horakihor
horakihor enabled auto-merge (squash) November 9, 2026 10:15
The DCO app missed the previous push during the GitHub webhooks incident.

Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@horakihor
horakihor merged commit 610e6a0 into main Nov 9, 2026
9 checks passed
@horakihor
horakihor deleted the remove-workspace-fanout branch November 9, 2026 10:48
horakihor added a commit that referenced this pull request Nov 9, 2026
Drops the GitHub handler's workspaceStream field, which #89 removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
horakihor added a commit that referenced this pull request Nov 12, 2026
TFBuddy gets two apply modes. Manual Mode is today's flow for people.
Auto Mode applies a merge request opened by a listed automation user once
it is approved, through the same apply path, then auto-merges it. It is
off unless TFBUDDY_AUTO_MODE_ENABLED is set and the repo's .tfbuddy.yaml
on the default branch has an autoMode section.

- The class comes from the MR author, every commit's author and the
  changed files, and each class lists the workspaces it may touch.
  Changes to .async-agent/, .gitlab/ or any .gitlab-ci.yml never run in
  Auto Mode.
- Auto Mode runs get a message built from fixed fields only:
  MR [!<iid>] [tfbuddy-class=<class>] sha=<head SHA> author=<user>, and
  each run checks out the exact commit it names.
- TFC/class is decided from the recorded plan outcome of every affected
  workspace, with one summary comment per MR.
- The approved event starts the existing apply path only when the MR is
  approved, every TFC/* status TFBuddy posted is green, and the pipeline
  and the configured required jobs succeeded.
- Every failure falls back to Manual Mode the same way: the MR is marked
  until it merges, TFC/class turns skipped, a comment is posted, and the
  head is planned again without the class. A fallen-back MR needs a
  person's approval to apply.

This squashes the 28 PoC commits and rebases them onto main after #82 to
#89. With the workspace fan-out gone (#89), the fan-out worker's Auto
Mode handling is dropped and the inline dispatcher's is kept. The two
Auto Mode buckets use main's retrying setup. TFBuddy's own MR events are
handled before the plan trigger is built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants