Skip to content

test: stabilize sub-millisecond scaling checks - #47

Open
usercao wants to merge 1 commit into
mainfrom
test/stabilize-scaling-clock
Open

usercao wants to merge 1 commit into
mainfrom
test/stabilize-scaling-clock

Conversation

@usercao

@usercao usercao commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Allow an individual text scaling case to select a larger input size, then raise N from 6,000 to 24,000 for four substitution and escape cases whose smaller samples can finish near one CPU clock tick.

On coarse or busy CI clocks, a sample near that boundary can be reported as 0.000ms or 1.000ms, making the measured 4N / N ratio unrepresentative or impossible to calculate. The larger fixtures keep those cases measurable while preserving the same ratio thresholds and expected output checks. All other text cases retain the existing default size.

Testing

  • corepack yarn test:scaling

The affected cases completed with measurable ratios inside the existing plausibility floor and scaling threshold.

Copilot AI balanced review requested due to automatic review settings October 30, 2026 03:30
@changeset-bot

changeset-bot Bot commented Oct 30, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f65ee46

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the scaling-guardrail input sizing changes.

No open issues 🎉

What changed in this PR

This PR stabilizes the performance scaling guardrail for fast substitution and escape cases by increasing their input sizes to reduce timing distortion on CI.

Changes:

  • Adds an optional per-case input size, keeping the 6,000 default.
  • Uses 24,000 for four fast cases while preserving existing ratio thresholds and output checks.
File Description
test/​performance/​scaling-check.ts Supports per-case input sizes and enlarges four fixtures for more reliable timing.

Review effort: Balanced

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

A test-only change to test/performance/scaling-check.ts that stabilizes the scaling guardrail against coarse CPU-clock quantization.

  • Per-case input size — TextCheckCase gains an optional n, threaded through defineTextCheck as a trailing parameter that defaults to SUBSTITUTION_CHECK_N (6,000), so existing cases keep their current size.
  • Larger fixtures for four fast cases — COARSE_CLOCK_TEXT_CHECK_N (24,000) raises N for the three substitution cases and one escape case whose 6,000-sized runs sit at roughly one clock tick, moving their N/4N samples to ~5ms/~20ms and out of the zero-or-one-tick range.
  • No threshold or expectation changes — ratio bounds, retry/early-exit logic, and expected-result checks are untouched, so the guardrail still distinguishes linear from quadratic behavior.

I verified the four affected checks pass locally with ratios around 4-5, and the remaining text cases already measure at least ~2ms at N, so the selection covers the near-tick cases on this host. format:check, typecheck, lint, and the PR CI are green.

Pullfrog  | View workflow run | Using big-pickle (free) | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants