Repository navigation
refactor!: remove workspace fan-out - #89
Merged
Merged
Conversation
Workspace fan-out (TFBUDDY_WORKSPACE_FANOUT_ENABLED, added in #62) published one message per workspace to the TFBUDDY_WORKSPACE_TRIGGERS stream and ran each one in a separate workspace worker. It was meant to stop a slow merge request from outliving the hook message's AckWait. Since #83 the hook queues are pull consumers that keep a message in progress while it runs, and the WORKSPACE_DISPATCHES claim skips workspaces an earlier delivery already dispatched. The inline path covers the problem fan-out was added for. Zapier's deployment has run with fan-out disabled since v0.6.0, after two rollbacks in May 2026. Keeping it means a second dispatch path that nothing in production exercises. Remove the workspace stream and worker, the TriggerTFCEvents enqueue path, the SetWorkspaceStream plumbing through the GitLab and GitHub hook handlers, and the workspace-fanout-enabled and workspace-jetstream-replicas settings. Every workspace now runs the way it did with fan-out disabled. Docs no longer describe fan-out, and say how to delete the stream that older releases left behind. Signed-off-by: Ihor Horak <ihor.horak@zapier.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change to inline-only workspace dispatch.
No open issues 🎉
1 file not reviewed
pkg/mocks/mock_tfc_trigger.go: Generated file
What changed in this PR
This PR removes workspace fan-out and makes inline dispatch the only execution path. It relies on the hook-consumer keep-alives and dispatch claims introduced in #83 to handle long-running events and prevent duplicate workspace runs.
Changes:
- Removes the workspace queue, worker, associated tests, and hook-handler wiring.
- Dispatches all touched workspaces from one checkout while preserving existing inline error handling and commit statuses.
- Removes fan-out settings and updates documentation, including cleanup guidance for the retired stream. Existing command lines using the removed flags must be updated.
| File | Description |
|---|---|
README.md |
Describes inline workspace dispatch and hook keep-alives. |
pkg/vcs/github/hooks/stream_worker.go |
Removes workspace-stream injection into GitHub triggers. |
pkg/vcs/github/hooks/stream_types.go |
Clarifies the retained delivery-ID comment. |
pkg/vcs/github/hooks/github_hooks_handler.go |
Removes the workspace publisher field and constructor argument. |
pkg/tfc_trigger/workspace_worker.go |
Deletes the dedicated workspace worker. |
pkg/tfc_trigger/workspace_worker_test.go |
Deletes workspace-worker tests. |
pkg/tfc_trigger/workspace_stream.go |
Deletes the workspace stream and message implementation. |
pkg/tfc_trigger/workspace_stream_test.go |
Deletes stream and fan-out tests. |
pkg/tfc_trigger/tfc_trigger.go |
Replaces selectable dispatch paths with direct inline execution. |
pkg/tfc_trigger/interfaces.go |
Removes SetWorkspaceStream from the trigger interface. |
pkg/runstream/workspace_dispatch.go |
Updates the dispatch-claim TTL comment. |
pkg/mocks/mock_tfc_trigger.go |
Removes the obsolete mock method. |
pkg/hooks/server.go |
Removes workspace-stream startup, worker setup, and health check. |
pkg/gitlab_hooks/stream_msgs.go |
Clarifies delivery-ID comments without changing behavior. |
pkg/gitlab_hooks/mr_actions.go |
Removes workspace-stream injection into MR triggers. |
pkg/gitlab_hooks/hook_worker.go |
Removes the worker’s workspace publisher dependency. |
pkg/gitlab_hooks/handler.go |
Removes the publisher field and constructor argument. |
pkg/gitlab_hooks/comment_actions.go |
Removes workspace-stream injection into comment triggers. |
internal/config/config.go |
Removes fan-out settings and narrows dedup-window documentation. |
docs/usage.md |
Updates generated configuration documentation. |
docs/contributing.md |
Updates the explanation for race-enabled testing. |
docs/architecture.md |
Documents inline processing and retired-stream cleanup. |
Review effort: Balanced
chpatinos
approved these changes
Nov 9, 2026
horakihor
enabled auto-merge (squash)
November 9, 2026 09:49
horakihor
disabled auto-merge
November 9, 2026 10:15
The DCO app did not report on the merge commit from "Update branch", so the required DCO check never arrived. Signed-off-by: Ihor Horak <ihor.horak@zapier.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
horakihor
enabled auto-merge (squash)
November 9, 2026 10:15
The DCO app missed the previous push during the GitHub webhooks incident. Signed-off-by: Ihor Horak <ihor.horak@zapier.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
horakihor
added a commit
that referenced
this pull request
Nov 9, 2026
Drops the GitHub handler's workspaceStream field, which #89 removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
horakihor
added a commit
that referenced
this pull request
Nov 12, 2026
TFBuddy gets two apply modes. Manual Mode is today's flow for people. Auto Mode applies a merge request opened by a listed automation user once it is approved, through the same apply path, then auto-merges it. It is off unless TFBUDDY_AUTO_MODE_ENABLED is set and the repo's .tfbuddy.yaml on the default branch has an autoMode section. - The class comes from the MR author, every commit's author and the changed files, and each class lists the workspaces it may touch. Changes to .async-agent/, .gitlab/ or any .gitlab-ci.yml never run in Auto Mode. - Auto Mode runs get a message built from fixed fields only: MR [!<iid>] [tfbuddy-class=<class>] sha=<head SHA> author=<user>, and each run checks out the exact commit it names. - TFC/class is decided from the recorded plan outcome of every affected workspace, with one summary comment per MR. - The approved event starts the existing apply path only when the MR is approved, every TFC/* status TFBuddy posted is green, and the pipeline and the configured required jobs succeeded. - Every failure falls back to Manual Mode the same way: the MR is marked until it merges, TFC/class turns skipped, a comment is posted, and the head is planned again without the class. A fallen-back MR needs a person's approval to apply. This squashes the 28 PoC commits and rebases them onto main after #82 to #89. With the workspace fan-out gone (#89), the fan-out worker's Auto Mode handling is dropped and the inline dispatcher's is kept. The two Auto Mode buckets use main's retrying setup. TFBuddy's own MR events are handled before the plan trigger is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Ihor Horak <ihor.horak@zapier.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This removes workspace fan-out (
TFBUDDY_WORKSPACE_FANOUT_ENABLED, added in #62). With fan-out on, a hook event published one message per workspace to theTFBUDDY_WORKSPACE_TRIGGERSstream, and a separate workspace worker ran each one. Fan-out was added so that a merge request touching many workspaces would not outlive the hook message'sAckWaitand get planned twice.#83 solved that problem on the inline path. The hook queues are now pull consumers that keep a message in progress while it runs, and the
WORKSPACE_DISPATCHESclaim skips workspaces that an earlier delivery already dispatched. Since then, fan-out has only been a second dispatch path that nothing exercises:TFBUDDY_WORKSPACE_FANOUT_ENABLED=falsesince v0.6.0 (September 2026). Fan-out ran in production twice in May 2026, on v0.5.0 and v0.5.1, and was rolled back both times. Nothing has been published toTFBUDDY_WORKSPACE_TRIGGERSsince 2026-05-22.What is removed:
pkg/tfc_trigger/workspace_stream.go(the stream, its message type and itsjsqueuepull queue) andworkspace_worker.go, with their tests.enqueuebranch inTriggerTFCEvents.dispatchWorkspacesnow callstriggerRunForWorkspacedirectly, with the same log line, error text and commit status as the old inline dispatcher.SetWorkspaceStreamand theWorkspacePublisherinterface, plus the field and constructor argument that carried them through the GitLab and GitHub hook handlers. TheTriggermock is regenerated.workspace-fanout-enabledandworkspace-jetstream-replicassettings.docs/.docs/usage.mdis regenerated. The chart never set either setting, so it has nothing to remove.DeliveryIDstays, because the dispatch claim and the auto-merge apply generation use it. Only its comments changed.Behaviour change for other deployments
Fan-out was on by default, and the README and docs presented it as the normal path. A deployment that never set
TFBUDDY_WORKSPACE_FANOUT_ENABLEDhas been running fan-out, and after this change it plans every workspace inline in the hook worker. That is the same behaviour asTFBUDDY_WORKSPACE_FANOUT_ENABLED=falsetoday.TFBUDDY_WORKSPACE_FANOUT_ENABLEDandTFBUDDY_WORKSPACE_JETSTREAM_REPLICASenvironment variables are now ignored.--workspace-fanout-enabledand--workspace-jetstream-replicasflags no longer exist, so a command line that still passes them fails to start.tfc planre-runs them.The title is marked breaking (
refactor!:), so semantic-release cuts v1.0.0 as the next release.After deploying
TFBuddy no longer creates or reads
TFBUDDY_WORKSPACE_TRIGGERS. Once no older release is running, the stream and its leftover push consumer (tfbuddy_workspace_trigger_worker) can be deleted from NATS:Deleting the stream also deletes the consumer. This PR does not delete anything itself.
Testing
go vet ./...: clean.go test -race ./...: all 17 packages pass.go generate ./internal/config/regeneratesdocs/usage.md, andmockgenregeneratespkg/mocks/mock_tfc_trigger.go. The only change in the mock is the removed method.🤖 Generated with Claude Code