Skip to content

docs(dsql): add CW cost alerts, fix CloudWatch AAS averaging - #295

Open
praba2210 wants to merge 1 commit into
mainfrom
docs/dsql-cloudwatch-cost-alerts
Open

praba2210 wants to merge 1 commit into
mainfrom
docs/dsql-cloudwatch-cost-alerts

Conversation

@praba2210

@praba2210 praba2210 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Adds DPU cost alarm guidance to the DSQL skill and fixes how Workflow 12 averages Average Active Sessions (AAS).

Related

The evals below were graded with the eval runner from #296; main's runner can't grade them on the current Claude CLI.

Changes

  • New references/cost-alerts.md: a per-cluster hourly TotalDPU alarm sized from a monthly budget, sent through an SNS topic encrypted with a customer managed key, plus a delivery test. The agent looks up the existing topic, key and alarm first, asks once before changing anything, and merges into existing policies instead of replacing them.
  • SKILL.md: routes cost questions and cost alarm problems to cost-alerts.md.
  • System diagnostics:
    • AAS is averaged with sum_over_time. Phase 1 shares come from one instant query per window, so nothing is summed by hand.
    • Phase 4 makes one batched metrics call per window with a 1-hour period. A 1-minute period returns no data for windows more than 15 days old.
    • Adds an AWS CLI / Query Studio fallback for when the CloudWatch MCP server is disabled.
  • mcp/mcp-setup.md: links the PromQL Region list instead of a stale copy.
  • Evals: adds evals 29–32 and one trigger case. Version 1.10.0 → 1.11.0.

Verification

  • mise run build passes.
  • Ran the alarm setup and delivery test on a test account; the email arrived.
  • Checked the PromQL and metrics changes against a live cluster.
  • Evals 29–32, three runs with and without the skill (Claude CLI 2.1.318): 52 of 53 vs 41 of 53 graded expectations passed. Without the skill, answers skipped the customer managed key (0 of 3) and the key policy merge (0 of 3).

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

Copilot AI balanced review requested due to automatic review settings November 4, 2026 18:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@praba2210
praba2210 force-pushed the docs/dsql-cloudwatch-cost-alerts branch from b3eaf23 to 5798e75 Compare November 4, 2026 21:35
Copilot AI balanced review requested due to automatic review settings November 4, 2026 21:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

The skill had no guidance for cost alerts, and the Workflow 12 PromQL
read only the latest one-minute sample, so a plain selector overcounted
sporadic wait events and instant queries ignored the window.

- Add cost-alerts.md: a per-cluster hourly TotalDPU alarm sized from a
  monthly budget, sent through an SNS topic on a customer managed KMS
  key, with a delivery test; TotalDPU already includes StreamDPU, and
  multi-Region budgets subtract storage in each peer Region
- Run the topic, key and alarm lookups first and ask once before any
  change; merge into existing key and topic policies instead of
  replacing them, and subscribe the email for existing topics too
- Average AAS with sum_over_time over each window or step, pass time
  at the window end, sort topk results, and keep span plus lookback
  within 7 days
- Take Phase 1 shares from one instant query per window and the
  intra-hour chunks from a 600s step, so nothing is summed by hand
- Batch the Phase 4 TotalTransactions and OccConflicts calls with a
  1-hour period; a 1-minute period returns nothing for windows that
  start more than 15 days ago
- Wait classes follow the AWS docs; check call count and latency
  before naming a cause of SingleRead growth
- Name CloudWatch Database Insights and add a fallback for when the
  CloudWatch MCP server is not enabled: AWS CLI for commit and OCC
  metrics, Query Studio or Database Insights for AAS
- Route cost questions and cost alarm problems to cost-alerts.md
- Link the CloudWatch PromQL Region list instead of a stale copy
- Fix "Active Average Sessions" to "Average Active Sessions"
- Add cost evals 29-32 and one trigger case
- Bump databases-on-aws to 1.11.0
@praba2210
praba2210 force-pushed the docs/dsql-cloudwatch-cost-alerts branch from 5798e75 to 7b98169 Compare November 5, 2026 03:36
Copilot AI balanced review requested due to automatic review settings November 5, 2026 03:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@praba2210
praba2210 marked this pull request as ready for review November 5, 2026 18:42
@praba2210
praba2210 requested review from a team as code owners November 5, 2026 18:42
@praba2210 praba2210 changed the title docs(dsql): add DPU cost alarms, fix CloudWatch AAS averaging docs(dsql): add CW DPU cost alarms, fix CloudWatch AAS averaging Nov 5, 2026
@praba2210 praba2210 changed the title docs(dsql): add CW DPU cost alarms, fix CloudWatch AAS averaging docs(dsql): add CW cost alerts, fix CloudWatch AAS averaging Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants