Repository navigation
fix: stabilize PullPreview OP 18-rc deploys and NC 35 smoke E2E - #91
Merged
Merged
Conversation
- Increased memory requests and limits for the op-buildsource-job to prevent OOM issues during Angular builds. - Added diagnostics to capture and report OOMKilled reasons for the op-buildsource-job in the collect-diagnostics script. - Implemented a function to check for OOMKilled status in helmfile-sync.sh, improving error handling during Helmfile sync. - Updated helmfile.yaml.gotmpl to ensure jobs complete before dependent services start, reducing the risk of memory pressure.
openproject:*-slim lacks git, so branch builds aborted silently under set -e. Also capture openproject-seeder and wait-for-db logs on PullPreview deploy failure.
BackoffLimitExceeded deletes seeder pods before final diagnostics; watch the Job during helmfile sync and capture stderr on first container failure.
Current logs can be from a restart mid-flight; --previous keeps the failed attempt's stderr after OnFailure restarts.
Seeder dies mid "Executing database migrations..." under the 1Gi limit on PullPreview; give it the same headroom as the web process.
--tail=200 only kept the pending-migration name flood and "Database seed failed with status 1"; drop the limit so the real exception is in CI logs. Also describe seeder pods.
Memory bump did not fix seed exit 1. --all-containers --previous fails on check-db-ready and hides the Ruby exception; capture seeder container logs only and print an error-line extract.
Label selectors default to 10 lines, which hid the Ruby seed exception behind the last migration names. Force full logs.
Helm starts seeder and web/worker together; rails runner in wait-for-db creates ar_internal_metadata before structure.sql loads. Override migrate to wipe an uninitialized public schema, and wait-for-db to poll via psql.
Header search is an inline combobox on NC 35; drop the Vue2 __vue__.onSubmit smart-picker hack; serialize personal-settings saves so dual toggles do not clobber unified search.
OP results live under unified-search-modal-* classes / listbox roles, not .unified-search-modal. Select smart-picker options via element handle before vue-select detaches the row.
Playwright clicks never fire vue-select option:selected on NC 35. Dispatch the same submit CustomEvent WorkPackagePickerElement uses so NcCustomPickerElement closes the modal and inserts the WP link.
Reused k3s nodes keep ValidatingAdmissionPolicy safe-upgrades.gateway.networking.k8s.io, which blocks Traefik's bundled Gateway CRDs (< v1.5.0). Ingress-only stack does not need them.
helmfile 0.170.1 rejects skipCRDs. Reused k3s nodes can keep safe-upgrades.gateway.networking.k8s.io, which blocks Traefik ~39 CRDs.
SQUASH_TM_SKIP_MISSING_AUTH only skipped a missing API token, so CI with a token but empty SQUASH_TM_ITERATION_ID still failed a green Playwright run. Treat any missing auth/target env the same.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
https://community.openproject.org/wp/TA-53
What are you trying to accomplish?
Make PullPreview E2E reliable for OpenProject 18.0-rc (git buildsource) + Nextcloud 35 smoke runs.
Deploy was failing or flaking on reused preview nodes (op-buildsource OOM / incomplete diagnostics, seeder schema:load racing web wait-for-db, Traefik Gateway CRD admission leftovers). After deploy succeeded, NC 35 UI changes broke unified search, personal-settings dual toggles, and smart-picker selection. Ad-hoc CI without a Squash iteration id then failed a green Playwright run on publish.
What approach did you choose and why?
Buildsource / seeder: Keep waitForJobs DAG ordering and richer seeder/buildsource diagnostics; fix the real seeder race by wiping stale migrate markers and making wait-for-db wait via psql (mounted through op-entry-configmap). Reverted a temporary seeder memory bump once root cause was clear.
Traefik on reused k3s: Dropped skipCRDs (unsupported on pinned helmfile 0.170.1). Before helmfile sync, delete leftover Gateway ValidatingAdmissionPolicy safe-upgrades.gateway.networking.k8s.io - Ingress-only stack doesn’t need Gateway CRDs.
NC 35 E2E: Update unified-search locators; serialize personal-settings saves; select smart-picker results by dispatching the same submit CustomEvent the Vue app uses (Playwright option click alone does not fire @option:selected).
Squash publish: Extend SQUASH_TM_SKIP_MISSING_AUTH so a missing iteration id soft-skips publish instead of failing CI (token-only was already soft-skipped).
Alternatives considered: bumping helmfile for skipCRDs, raising seeder memory permanently, Vue2 vue.onSubmit hacks - discarded for version lock-in, masking races, and brittle private API use.
AI involvement
Directed – I specified the requirements and AI implemented most of it; I validated via testing rather than a full line-by-line review.
Merge checklist