Repository navigation
Fix concurrent Lambda invocation trace isolation - #761
Conversation
All Build and Lint jobs fail at the install step because the committed lockfile pins semver@6.3.1, while the ^6.3.0 range now resolves to semver@6.3.2 (published upstream since the lockfile was last updated): Invalid: lock file's semver@6.3.1 does not satisfy semver@6.3.2 Regenerate the three semver entries to 6.3.2 so package.json and package-lock.json are back in sync. No other dependencies change and lockfileVersion stays 2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #761 +/- ##
==========================================
+ Coverage 84.31% 85.24% +0.92%
==========================================
Files 36 36
Lines 1817 1843 +26
==========================================
+ Hits 1532 1571 +39
+ Misses 285 272 -13 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| if (cls_mode) { | ||
| var segment = contextUtils.getNamespace(NAMESPACE).get(SEGMENT); | ||
| var namespace = contextUtils.getNamespace(NAMESPACE); | ||
| var invocation = process.env.LAMBDA_TASK_ROOT && require('./env/aws_lambda').getInvocationContext(); |
There was a problem hiding this comment.
This reads process.env.LAMBDA_TASK_ROOT on every getSegment() call, and setSegment() does the same on line 124. Both are on the hot path and process.env lookups aren't free, so non-Lambda users pay for this too. Could you resolve it once at module load (for example a module-level const isLambda = !!process.env.LAMBDA_TASK_ROOT) and reuse it in both places?
There was a problem hiding this comment.
require('./env/aws_lambda') also runs on every getSegment()/setSegment() call, including every .then() when capturePromise() is enabled. Can you also cache the module lazily on first use?
| if (active) { | ||
| segments.set(active, current); | ||
| } else { | ||
| segment = current; |
There was a problem hiding this comment.
When there's no active CLS context, this replaces the invocation's fallback segment, so every later getSegment() in this invocation without a mapped slot returns current instead of the facade. Before this change, setSegment() without an active context failed and logged "Failed to set the current sub/segment on the context." without changing anything. Unless there's a case that needs this, could it keep the old behavior, e.g. return false here so contextUtils.setSegment() logs the same warning, instead of silently swapping the invocation's root?
jj22ee
left a comment
There was a problem hiding this comment.
ty for this fix! added a couple small requests.
| if (cls_mode) { | ||
| var segment = contextUtils.getNamespace(NAMESPACE).get(SEGMENT); | ||
| var namespace = contextUtils.getNamespace(NAMESPACE); | ||
| var invocation = process.env.LAMBDA_TASK_ROOT && require('./env/aws_lambda').getInvocationContext(); |
There was a problem hiding this comment.
require('./env/aws_lambda') also runs on every getSegment()/setSegment() call, including every .then() when capturePromise() is enabled. Can you also cache the module lazily on first use?
Cache the Lambda runtime check and load the context helper lazily. Preserve the invocation facade and log the existing warning when no CLS context is active. Test the classic runtime's persistent InvokeStore and isolate Lambda tests so the runtime flags are set before the SDK loads. Assisted-by: OpenAI Codex
Fixes #760.
Concurrent Lambda Managed Instance invocations can share an active CLS segment and facade. This can attach annotations to another invocation, emit a subsegment on the wrong trace, or drop sampled work after an unsampled invocation resolves the shared facade.
Use the runtime's InvokeStore context identity to give each invocation its own facade and CLS segment slots. The slots follow the CLS prototype chain so nested and parallel capture scopes restore their caller. Weak keys let expired contexts and tracing state be collected. Startup initializes the legacy CLS slot, and runtimes without an invocation context use that fallback.
The context helpers cache Lambda detection at module load and load the Lambda module on first use. An attempt to set a segment without an active CLS context logs the existing warning and preserves the invocation facade.
The isolation suite uses the runtime's InvokeStore 0.2.2. It covers concurrent invocations and classic Lambda, whose single-context store retains the previous context between handlers. The classic test leaves a subsegment assigned in one handler and verifies that the next gets a fresh facade and emits on its own trace. The missing-context test verifies the warning and exercises capture in a new CLS scope.
Local validation
The reporter's unchanged handler reproduced cross-attribution before the initial isolation patch. With that patch, the local replay emitted 30 correctly attributed documents in each of the three modes. The initial patch's separate 10,000-invocation stress run observed zero attribution errors and emitted the expected 13,334 sampled documents.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.