Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@
"serverless",
"postgresql"
],
"version": "1.10.0"
"version": "1.11.0"
},
{
"category": "deployment",
Expand Down
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@
"license": "Apache-2.0",
"name": "databases-on-aws",
"repository": "https://github.com/awslabs/agent-plugins",
"version": "1.10.0"
"version": "1.11.0"
}
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "databases-on-aws",
"version": "1.10.0",
"version": "1.11.0",
"description": "Expert database guidance for the AWS database portfolio. Design schemas, execute queries, handle migrations, and choose the right database for your workload.",
"author": {
"name": "Amazon Web Services",
Expand Down
11 changes: 5 additions & 6 deletions plugins/databases-on-aws/skills/dsql/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: dsql
description: "Build with Aurora DSQL — manage schemas, execute queries, handle migrations, diagnose query plans, diagnose cluster performance, load data, and develop applications with a serverless, distributed SQL database. Covers IAM auth, multi-tenant patterns, MySQL-to-DSQL and PostgreSQL-to-DSQL schema conversion, foreign key constraints, OCC retry patterns, ORM migration (Django/EF Core/Hibernate/Rails/SQLAlchemy), DDL operations, query plan explainability, system diagnostics via CloudWatch AAS, SQL compatibility validation, and bulk data loading. Load before answering DSQL limit or error-code questions; it has limits docs omit. Triggers on phrases like: DSQL, Aurora DSQL, distributed SQL database, serverless PostgreSQL-compatible database, migrate to DSQL, DSQL query plan, DSQL EXPLAIN ANALYZE, DSQL ENUM, DSQL foreign key, DSQL OCC retry, DSQL multi-region, DSQL JSONB, DSQL GIN index, load into DSQL, load CSV into DSQL, bulk load DSQL, aurora-dsql-loader, DSQL slow, DSQL performance, DSQL wait events, DSQL AAS."
description: "Build with Aurora DSQL — manage schemas, execute queries, handle migrations, diagnose query plans, diagnose cluster performance, load data, and develop applications with a serverless, distributed SQL database. Covers IAM auth, multi-tenant patterns, MySQL-to-DSQL and PostgreSQL-to-DSQL schema conversion, foreign key constraints, OCC retry patterns, ORM migration (Django/EF Core/Hibernate/Rails/SQLAlchemy), DDL operations, query plan explainability, CloudWatch AAS diagnostics and cost alarms, SQL compatibility validation, and bulk data loading. Load before answering DSQL limit or error-code questions; it has limits docs omit. Triggers on phrases like: DSQL, Aurora DSQL, distributed SQL database, serverless PostgreSQL-compatible database, migrate to DSQL, DSQL query plan, DSQL EXPLAIN ANALYZE, DSQL ENUM, DSQL foreign key, DSQL OCC retry, DSQL multi-region, DSQL JSONB, DSQL GIN index, load into DSQL, load CSV into DSQL, bulk load DSQL, aurora-dsql-loader, DSQL slow, DSQL performance, DSQL wait events, DSQL AAS."
license: Apache-2.0
metadata:
tags: aws, aurora, dsql, distributed-sql, distributed, distributed-database, database, serverless, serverless-database, postgresql, postgres, sql, schema, migration, multi-tenant, iam-auth, aurora-dsql, mcp, orm, enum, foreign-key, occ-retry, django, ef-core, dotnet, csharp, hibernate, rails, multi-region, schema-conversion, type-mapping, data-loading, system-diagnostics, wait-events, aas, performance, cloudwatch
Expand Down Expand Up @@ -29,6 +29,7 @@ Load these files as needed for detailed guidance:
| [dsql-examples.md](references/dsql-examples.md) | Load for implementation examples | Multi-tenant access, batch operations, identity and sequences, connection pooling |
| [onboarding.md](references/onboarding.md) | User requests "Get started with DSQL" | Interactive step-by-step guide |
| [occ-retry-patterns.md](references/occ-retry-patterns.md) | MUST load for OCC retry code or conflict mitigation | Connectors, `40001` retry, non-retryable `23503`, FK read conflicts, idempotent design |
| [cost-alerts.md](references/cost-alerts.md) | Setting up or debugging cost alarms, or explaining or reducing a cluster's cost | `TotalDPU` alarm, hourly threshold from a monthly budget, SNS notification |

### MCP:

Expand Down Expand Up @@ -189,7 +190,7 @@ When the user reports a performance problem, use this table to select the correc
| General performance complaint, "cluster is slow", "something changed", latency regression, no specific query identified | **Workflow 12** (System Diagnostics) — observe via CloudWatch first |
| Specific query or query_id to investigate, "explain this plan", "why is this query slow" | **Workflow 9** (Query Plan Explainability) — direct EXPLAIN analysis |
| OCC conflicts, commit errors, retry storms | **Workflow 12** (System Diagnostics) — confirm via CW metrics before investigating |
| Cost optimization, "where is compute time spent" | **Workflow 12** (System Diagnostics) — identify top contributors first |
| "Where is compute time spent" | **Workflow 12** (System Diagnostics) — identify top contributors first |

**Rule:** When in doubt, start with Workflow 12. It identifies specific queries to investigate and routes to Workflow 9 with context.

Expand Down Expand Up @@ -276,11 +277,9 @@ MUST load [pg-migrations/type-mapping.md](references/pg-migrations/type-mapping.

Load [orm-guides/overview.md](references/orm-guides/overview.md) for adapter names, locking guidance, and framework-specific gotchas.

### Workflow 12: System Diagnostics (CloudWatch AAS)
### Workflow 12: System Diagnostics (CloudWatch Database Insights)

Diagnose cluster performance by querying `db.active_sessions.avg` via PromQL. Detects temporal anomalies in wait event distribution, identifies regressed queries, and routes to Workflow 9 for per-query investigation.

**Requires:** CloudWatch MCP server (`awslabs.cloudwatch-mcp-server`) enabled and configured with PromQL access in the same region as the cluster — see [mcp/mcp-setup.md](mcp/mcp-setup.md#cloudwatch-mcp-server-system-diagnostics--workflow-12) for enabling it, region requirements, and the session restart needed for its tools to register.
Diagnose cluster performance by querying `db.active_sessions.avg` via PromQL. Detects temporal anomalies in wait event distribution, identifies regressed queries, and routes to Workflow 9 for per-query investigation. The CloudWatch MCP server (`awslabs.cloudwatch-mcp-server`) is optional — without it, or if its tools are not callable, the workflow falls back to the AWS CLI and CloudWatch Query Studio; never guess AAS values. To enable it with PromQL access in the cluster's region, see [mcp/mcp-setup.md](mcp/mcp-setup.md#cloudwatch-mcp-server-system-diagnostics--workflow-12).

MUST load [system-diagnostics/workflow.md](references/system-diagnostics/workflow.md) at entry — it defines prerequisites, 5 diagnostic phases, temporal baselines, and the routing to Workflow 9 for identified queries.

Expand Down
11 changes: 5 additions & 6 deletions plugins/databases-on-aws/skills/dsql/mcp/mcp-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,12 @@ has custom AWS configurations or would like to allow/disallow the MCP server mut

## CloudWatch MCP Server (System Diagnostics — Workflow 12)

Workflow 12 (System Diagnostics) reads Aurora DSQL Active Average Sessions (AAS) telemetry
Workflow 12 (System Diagnostics) reads Aurora DSQL Average Active Sessions (AAS) telemetry
through the CloudWatch MCP server's PromQL tools. This is a **separate server** from
`aurora-dsql` — it reads CloudWatch metrics, not the database — so it has its own entry in
`.mcp.json`. The plugin ships it disabled because it needs region and credential details the
plugin can't know in advance.
plugin can't know in advance. Without it, the workflow uses the AWS CLI for commit and OCC
metrics and asks the user to run the PromQL in CloudWatch Query Studio.

The plugin's `.mcp.json` pre-configures it as:

Expand Down Expand Up @@ -154,10 +155,8 @@ To enable it:
**Region matters.** Each query must target the region where the cluster's metrics live (the
cluster's own region). Set `AWS_REGION` to that region for the default, and/or pass `region`
explicitly on each tool call — the PromQL tools accept a per-call `region`, so one running server
can serve clusters in more than one region. Either way, CloudWatch PromQL is only available in a
subset of regions — at the time of writing: `us-east-1`, `us-west-2`, `eu-west-1`,
`ap-southeast-1`, `ap-southeast-2`. If a cluster is in a region not on this list, PromQL-based
diagnostics are not available for it; verify the current list in the CloudWatch documentation.
can serve clusters in more than one region. Either way, the cluster's region must support CloudWatch
PromQL — check the [supported Regions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-PromQL.html#CloudWatch-PromQL-Regions).

**Restart after enabling.** MCP tools are registered when the session starts. If you enable
the server (or fix its config) mid-session, its tools (`execute_promql_range_query`,
Expand Down
193 changes: 193 additions & 0 deletions plugins/databases-on-aws/skills/dsql/references/cost-alerts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
# Cost Alerts for Aurora DSQL

Alert when a cluster's hourly DPU usage nears the hourly share of a monthly budget. To explain or
reduce a cluster's cost, first compare the DPU components (`ComputeDPU`, `ReadDPU`, `WriteDPU`,
`MultiRegionWriteDPU`, `StreamDPU`) and `ClusterStorageSize` over the period; use Workflow 12
only if query activity drove it, then confirm the saving in `TotalDPU`. See
[How billing works in Aurora DSQL](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/billing-metering.html)
and [Create a CloudWatch alarm based on a static threshold](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ConsoleAlarms.html).

Ask the user for the cluster ID, its Region, the monthly budget in USD, whether that budget covers
only this cluster's DPU usage or also its storage and other Regions, the customer managed KMS key
to use (or whether to create one), and the notification email. Get `ACCOUNT_ID` from
`aws sts get-caller-identity`. First run the read-only lookups in step 1 and the `describe-alarms`
in step 3. Before creating or changing anything, **MUST** show the account, Region, KMS key, topic
(new or existing, and its subscribers), each policy change (the full merged policy; for a new key,
the default policy plus the statement), email, threshold and any existing alarm's settings, and
wait for the user's go-ahead.

## Metric

`TotalDPU` in namespace `AWS/AuroraDSQL`, dimension `ClusterId`, is the cluster's billed DPU
usage. Storage is billed separately and is not included. **MUST** use the `Sum` statistic for DPU
metrics; use `Average` for `ClusterStorageSize`.

## Threshold

The alarm evaluates one hour at a time, so the threshold is in DPUs per hour and comes from the
part of the monthly budget, in USD, left for this cluster's DPU usage. If the budget also covers
storage, first subtract the storage cost (`ClusterStorageSize`, reported in bytes, converted to
GiB x the Region's per-GiB-month rate). A multi-Region cluster is billed for storage in each peer
Region, so when the budget also covers the other Regions, subtract each peer's storage at that
Region's rate. If it is shared across Regions, split the remainder between them; writes
replicated to peers are billed as `MultiRegionWriteDPU` in the Region where they originate, so the
writing Region needs the larger share.

```
threshold = DPU budget (USD) / per-DPU rate (USD per DPU) / 730 hours x 0.8
```

The per-DPU rate varies by Region — get it from
[Aurora DSQL pricing](https://aws.amazon.com/rds/aurora/dsql/pricing/), which lists it per million
DPUs (for example, $8 per million DPUs is 0.000008 USD per DPU).

## Create the Alarm

1. Create an encrypted SNS topic. Alarm messages carry the account ID and cluster ID, so the topic
**MUST** use SSE-KMS (at rest) with a customer managed key, which has a monthly charge (see
[AWS KMS pricing](https://aws.amazon.com/kms/pricing/)). The AWS managed key `alias/aws/sns`
does not work: CloudWatch cannot use it, and the alarm fails to notify. `create-topic` returns
an existing topic of the same name without an error, so first check for one:

```bash
aws sns get-topic-attributes --region REGION \
--topic-arn arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts
aws sns list-subscriptions-by-topic --region REGION \
--topic-arn arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts
```

Resolve each existing key (the user's and the topic's `KmsMasterKeyId`, which may be an alias)
with `aws kms describe-key --region REGION --key-id KEY`; use its `KeyMetadata.Arn` as
`KMS_KEY_ARN` (`get-key-policy` rejects aliases). If `KeyManager` is not `CUSTOMER`, or the
topic has another key or none, stop and ask the user.

For a new key, after the go-ahead, use the `KeyMetadata.Arn` from
`aws kms create-key --region REGION --description "DSQL cost alarms"` as `KMS_KEY_ARN`. Either
way, add this statement to the key's policy. For this and the topic policy below, keep every
existing statement, skip the statement if Sid `AllowDsqlCostAlarm` already allows the alarm, and
write only the merged policy the user approved — both writes replace the whole policy:

```json
{
"Sid": "AllowDsqlCostAlarm",
"Effect": "Allow",
"Principal": { "Service": "cloudwatch.amazonaws.com" },
"Action": ["kms:Decrypt", "kms:GenerateDataKey*"],
"Resource": "*",
"Condition": {
"StringEquals": { "aws:SourceAccount": "ACCOUNT_ID" },
"ArnLike": {
"aws:SourceArn": "arn:aws:cloudwatch:REGION:ACCOUNT_ID:alarm:DSQL-Hourly-DPU-Budget-Warning-*"
}
}
}
```

```bash
aws kms get-key-policy --region REGION --key-id KMS_KEY_ARN --policy-name default \
--query Policy --output text > "${TMPDIR:-/tmp}/dsql-key-policy.json"
# Merge the statement above into the file; after the go-ahead:
aws kms put-key-policy --region REGION --key-id KMS_KEY_ARN --policy-name default \
--policy "file://${TMPDIR:-/tmp}/dsql-key-policy.json"
rm "${TMPDIR:-/tmp}/dsql-key-policy.json"
```

If the topic does not exist, create it and allow only these alarms to publish:

```bash
aws sns create-topic --name dsql-cost-alerts --region REGION \
--attributes KmsMasterKeyId=KMS_KEY_ARN
aws sns set-topic-attributes --region REGION \
--topic-arn arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts \
--attribute-name Policy --attribute-value '{
"Version": "2012-10-17",
"Statement": [{
"Sid": "AllowDsqlCostAlarm",
"Effect": "Allow",
"Principal": {"Service": "cloudwatch.amazonaws.com"},
"Action": "sns:Publish",
"Resource": "arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts",
"Condition": {
"StringEquals": {"aws:SourceAccount": "ACCOUNT_ID"},
"ArnLike": {"aws:SourceArn": "arn:aws:cloudwatch:REGION:ACCOUNT_ID:alarm:DSQL-Hourly-DPU-Budget-Warning-*"}
}
}]
}'
```

If it exists, merge only this `sns:Publish` statement (not the KMS one) into its current `Policy`
the same way and write it with `set-topic-attributes`. In both cases, subscribe the email unless
it is already listed. The email **MUST** be a monitored address the user confirms, such as a team
or on-call list, not a personal address. The recipient must confirm the subscription.

```bash
aws sns subscribe --region REGION --protocol email --notification-endpoint EMAIL \
--topic-arn arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts
```

2. Create the alarm, named `DSQL-Hourly-DPU-Budget-Warning-CLUSTER_ID` to match the policies above.
`put-metric-alarm` overwrites an alarm of the same name, so each cluster needs its own. For
another cluster in the same Region, the topic and key already allow it: run the step 1 lookups,
then steps 2 and 3. In the console: cluster **Summary** tab, **Usage and cost** panel,
**Create alarm** — it prefills `TotalDPU`, the namespace and `ClusterId`; set **Sum**, a 1-hour
period, the threshold, **Missing data treatment** to not breaching, and the topic from step 1
(not a new one) for both the alarm and OK states. Or with the AWS CLI:

```bash
aws cloudwatch put-metric-alarm --region REGION \
--alarm-name DSQL-Hourly-DPU-Budget-Warning-CLUSTER_ID \
--namespace AWS/AuroraDSQL --metric-name TotalDPU \
--dimensions Name=ClusterId,Value=CLUSTER_ID \
--statistic Sum --period 3600 --evaluation-periods 1 --datapoints-to-alarm 1 \
--threshold THRESHOLD --comparison-operator GreaterThanThreshold \
--treat-missing-data notBreaching \
--alarm-actions arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts \
--ok-actions arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts
```

3. Verify the setup. `describe-alarms` must show the alarm on `CLUSTER_ID` with the topic in its
actions, and `list-subscriptions-by-topic` must show the email confirmed. For a user's own alarm,
find it with `describe-alarms-for-metric` (step 2's namespace, metric and dimensions), check
`ActionsEnabled`, and use its name and topic instead, also in each policy's `aws:SourceArn`. If
the email is still `PendingConfirmation`, ask the user to confirm it, then check again:

```bash
aws cloudwatch describe-alarms --region REGION \
--alarm-names DSQL-Hourly-DPU-Budget-Warning-CLUSTER_ID
aws sns list-subscriptions-by-topic --region REGION \
--topic-arn arn:aws:sns:REGION:ACCOUNT_ID:dsql-cost-alerts
```

Then test delivery now rather than at the first real breach. Ask the user before sending a test
email to every subscriber (and possibly an OK email when the alarm returns to its real state,
usually within a minute). Once they agree, force the alarm into `ALARM` with `set-alarm-state`,
which needs `cloudwatch:SetAlarmState`; if it is already `ALARM`, check its history instead (to
retest a fix, set `OK` first). `describe-alarm-history` since `START` must then show
`Successfully executed action` with `NewStateValue` `ALARM` in its `publishedMessage`. If the
entry is missing, rerun it with the same `START` every 10 seconds for up to 90 seconds, then
report to the user. An SNS or KMS authorization error means the key or topic policy from step 1
does not allow this alarm.

```bash
# Only after the user agrees to the test email
START=$(date -u +%Y-%m-%dT%H:%M:%SZ); echo "START=$START"
aws cloudwatch set-alarm-state --region REGION \
--alarm-name DSQL-Hourly-DPU-Budget-Warning-CLUSTER_ID \
--state-value ALARM --state-reason "Delivery test"
aws cloudwatch describe-alarm-history --region REGION \
--alarm-name DSQL-Hourly-DPU-Budget-Warning-CLUSTER_ID --history-item-type Action \
--start-date "$START"
```

Then tell the user what the alarm covers — this cluster's hourly DPU usage, including any CDC
streaming — and what it does not: storage, other Regions and month-to-date spend. It notifies
but does not cap spending.

## Limits

- The alarm checks single hours, not month-to-date spend: a spiky hour can alarm while the month
stays under budget. Use AWS Budgets for cumulative spend, including charges outside Aurora DSQL.
- Multi-Region clusters: repeat steps 1–3 in each Region — the key, topic and alarm are regional
— on that Region's cluster ID, with that Region's share of the budget.
- CDC: `TotalDPU` already includes `StreamDPU`, summed across all streams. For one stream's share,
read `StreamDPU` with dimensions `ClusterId` and `StreamId`.
Loading
Loading