Skip to content

feat: widgets, agent-drawn diagrams and charts people edit and comment on (collaboration M8) - #180

Open
maxsam4 wants to merge 19 commits into
stack/17-m10b-catchupfrom
stack/18-m8-widgets
Open

maxsam4 wants to merge 19 commits into
stack/17-m10b-catchupfrom
stack/18-m8-widgets

Conversation

@maxsam4

@maxsam4 maxsam4 commented Nov 19, 2026

Copy link
Copy Markdown

Replaces #159 (review history there).

One link of a linear stack, one milestone per PR. The base is stack/17-m10b-catchup (#179), so this diff is this milestone's own work plus the merge resolutions it needs against the links below it.

Migration: regenerated with drizzle-kit on this stack as 0026_widgets.sql (was 0023_widgets.sql).

Licences: new dependencies are @resvg/resvg-js@2.6.2 (MPL-2.0, used unmodified as a library, like BlockNote), quickjs-emscripten-core@0.32.0 and @jitl/quickjs-wasmfile-release-sync@0.32.0 (MIT), acorn@8.19.0 and acorn-jsx@5.3.2 (MIT), and roboto-fontface@0.10.0 (Apache-2.0, the font both the server and the page draw widgets with). All are pinned exactly and older than the 14-day release-age gate.

Summary

Agents draw diagrams and charts (widgets) in live documents, and people work on them like the rest of the document: they edit labels in place and comment on parts and data points.

  • The module. A widget is one JSX/SVG module: a bare <svg>, or export default function render({ rows, tokens }). It has no imports, no fetch, no style and no foreignObject, and its colours are theme tokens only.
    • Every element holding words the agent typed carries a literal data-text-id.
    • Named parts carry data-anchor.
    • Marks drawn from rows carry data-row and data-col.
  • One parser, and safe rendering on the server. acorn with acorn-jsx parses the module, and Primer's own emitter compiles JSX to __h() calls in place, so error lines are the agent's own.
    • The validator refuses imports, fetch, eval, globals, style, foreignObject, literal colours, and missing or computed ids. Each refusal comes with its line and a fix.
    • The module runs in a fresh QuickJS (WASM) runtime with no host functions: 16 MB, 50 ms, 5,000 elements, and no clock or random number source.
    • Its output always passes a runtime check: the sanitizer's SVG lists; one positive value grammar per attribute (isWidgetAttributeValue in api-types, linear and length-capped); unique literal text ids; rows that exist; 512 KB of SVG; a viewBox of at most 4096; and ids prefixed per widget.
    • PNGs come from resvg, at 1× and at most 2048 px.
  • Draft, preview, publish.
    • primer_widget_draft returns light and dark PNG previews as MCP image content.
    • primer_widget_publish publishes only if the draft still renders exactly the preview's hash, so a draft with errors never replaces the live widget.
    • primer_widget_get returns the source, rows, labels (with targets and versions, fenced) and a PNG of the live widget.
    • REST routes cover the same: draft, publish, get and image.
  • In the editor. A widget block draws its live SVG inline for the page's theme, through the block services, the same way M6's page view does.
    • Clicking a label opens the shared Overlay on widget/<blockId>/<textId>, to edit it or comment on it. PrimerThreadStore's scope now accepts a textId.
    • Clicking a mark drawn from a row comments on that data point (DataPointAnchor { row, column }); a named part takes a WidgetPartAnchor. Commented parts are marked and listed under the widget.
    • The server allowlists the SVG. The browser parses it once as XML, checks the whole tree (SVG namespace, element and text nodes only, the same allowlists and value grammar), and mounts it with importNode, never as markup.
  • Labels are fragments like page text and use M6's setPidText and readPidText.
    • primer_apply_ops update on { tabId, blockId, textId } edits a label by its version.
    • Suggestions keep the textId, as they keep a pid.
    • A republish carries labels three ways: an unchanged label keeps the person's words. Where both changed, the person's words stay and the agent is told (kept).
  • Records. A draft can draw a collection (collection instead of rows): its shared records become rows, with id set to the record id. A coalesced data.changed redraws the published widget, and a failed redraw keeps the last good publish.
  • Checkpoints (C11/C12).
    • Bundles keep widgets/<id>.light.svg, .dark.svg and .json.
    • static.html, export.html and the print view draw the SVG for both themes, and Markdown export keeps a fenced primer-widget reference.
    • Restore brings back the reachable sub-fragments, then republishes each widget row from the bundle.
    • Purge and failed-bundle cleanup delete every file in a bundle's manifest.
    • Every id read from the Y.Doc is validated, and every lookup by it is scoped by document and block.
  • Skill. widgets.md holds the rules, one house style (frame, type, grey first with one accent, shapes, fitting words) and copyable flows, trees, 2×2s, timelines, funnels, roadmaps, and bar, line and area charts. It is written after the quality bar of Claude Docs' widget guidance, in our own words.
    • Both MCP servers serve it as primer://skills/widgets, through the same loader as primer://skills/authoring, which now links to it.

Deviations from the plan

  • quickjs-emscripten-core plus the release-sync variant is used instead of quickjs-emscripten, which would also install the debug builds.
  • The widget block is placed at the first successful draft and shows "drawing" until the first publish.
  • Block props are { widgetId, hash, textIds }, not just { widgetId }. The hash makes viewers redraw on a publish, and textIds lets reachableFragments stay a pure function of the Y.Doc.
  • rows is split into rows (published) and draft_rows, with draft_collection alongside, so a pending draft never changes what the live widget draws.
  • Text computed from several rows (a total, tick numbers) needs no id or row. Typed words always need a literal id, and an element with an id holds only typed words.
  • primer_widget_get is a third tool.
  • MCP images ride an additive images field that both transports append, so existing handlers keep their text-only type.
  • An agent that may only suggest in a document is refused a draft (forbidden). A publish changes what people see at once, and there is no suggested-widget flow.
  • useThreadEvents now shares one EventSource per document. One stream per listener exhausted the browser's per-host connections and stalled widget image requests.

Test evidence

  • All six CI steps pass locally on the head (Postgres 17.9):
    • codegen-drift-check;
    • migrate;
    • typecheck;
    • build;
    • tests: api 1029 passed (plus 1 todo), web 369, mcp 125, blocks 3;
    • pnpm audit --audit-level=high.
  • The plan's tests are all present:
    • computed text ids are rejected;
    • fetch and imports are rejected;
    • serializer drops script, event handlers and external hrefs;
    • infinite loop is stopped by the CPU limit;
    • a tree over 5,000 nodes is refused before rasterising;
    • an output with a computed style is refused at runtime even if the source passed;
    • a draft with errors never replaces the live widget;
    • publish with a stale previewHash is refused;
    • label edit survives republish when the text id is kept;
    • data-point comment anchors to row and column;
    • every skill template validates and renders.
  • Further tests cover:
    • hostile attribute values, as one table run through both the server and the browser checkers (image-set(, src(, URL(, url (, escaped and entity forms);
    • a 100k-digit transform, refused at once;
    • hostile SVG in the browser (CDATA, a processing instruction, a comment, an HTML-namespace script, foreignObject, onload, xlink);
    • cross-document widget ids and traversal-shaped ids in bundles and restore;
    • label ops and suggestions by textId;
    • a chart redrawn when its collection changes;
    • checkpoint and restore.
  • Browser check (Chromium; a built API with mock auth, and the web app):
    1. An agent minted by a person created both widgets over /mcp. primer_widget_draft returned a text part and two images for each, and primer_widget_publish published them: a flowchart (the skill's flow) and a bar chart with inline rows.
    2. In the document, both widgets render at column width.
    3. A person clicked a label and typed in the overlay. The SVG shows "Build, lint and test first" live, and primer_widget_get returns it, fenced.
    4. A person clicked the week-3 bar and commented. The thread's anchor is { type: 'data-point', row: 'w3', column: 'signups' }, and it is listed under the chart as "w3 · signups".
    5. The PNG from primer_widget_get matches the page's render: same face, same layout, the edited label included. There were no console errors.

🤖 Generated with Claude Code

maxsam4 and others added 19 commits November 19, 2026 21:44
Widgets are agent-drawn SVG diagrams and charts. An agent's module is JSX or
a bare <svg>; one parser (acorn + acorn-jsx) reads it and Primer's own
emitter compiles JSX to __h() calls in place, keeping every other byte and
every line number, so errors point at the agent's own lines.

The static validator gives the agent a line and a fix (no imports, fetch,
eval or globals; no style or foreignObject; tokens instead of literal
colours; a literal data-text-id on typed words and nothing else under it).
It is not the safety boundary: the module runs in a fresh QuickJS (WASM)
runtime with no host functions, 16 MB, 50 ms and 5,000 elements, without
clock or random so a publish can re-render the exact preview. Its output
always passes the runtime check in serializeSvg: the sanitizer's SVG
allowlist, refusal of style, foreignObject, non-token colours, unknown or
repeated text ids and unknown rows, the node and 512 KB bounds, and ids
prefixed per widget.

PNG previews come from resvg (MPL-2.0) with Roboto (Apache-2.0) unpacked
from roboto-fontface's WOFF, so the server draws the face the page loads,
and a host without fonts still draws text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A widget is a `widget` block in the shared schema plus a `widgets` row. A
draft renders both themes, stores beside the live publish and returns PNG
previews with people's label edits applied; a publish re-renders the draft
and goes through only if it is exactly the preview the agent saw (its
previewHash), guarded against a concurrent publish. A draft with errors
never touches the live widget.

Label words live in the Y.Doc, one fragment per label
(`widget/<blockId>/<textId>`, one paragraph whose id is the block's), written
through applyPlan. A publish carries each label three ways against the
previous publish, so an agent's rewording lands where nobody edited and a
person's edit is never lost; the agent is told when its words lost to one.
The same plan writes the block's hash and text ids, so viewers redraw and
reachableFragments lists the labels.

The plan's single `rows` column is split into `rows` (published) and
`draft_rows`, so a pending draft never changes what the live widget draws.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agents draft, publish and read widgets with primer_widget_draft,
primer_widget_publish and primer_widget_get on both MCP servers, each backed
by a REST route that calls the same store function. A draft returns its
light and dark PNG previews as MCP image content, and a refusal carries each
module error's line and fix, so the agent can correct and redraw without
reading anything else. People's views fetch the live SVG (allowlisted markup,
served with a no-script CSP) from /api/widgets/:id/image.

Images ride an additive `images` field on tool results that the transports
append to `content`, so existing handlers and tests keep their text-only type.
The local server's widget calls read refusals as data in their own request
path, leaving the shared error mapping untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
widgets.md teaches agents to draw widgets people can work with: when a
picture beats prose, the module rules the validator enforces, the three kinds
of text (typed labels with ids, words drawn from rows, computed words), parts
and data points, one house style (frame, type, grey-first colour with one
accent, shapes, fitting words), the draft-look-publish loop, and copyable
flows, trees, 2x2s, timelines, funnels, roadmaps and bar, line and area
charts. A test renders every module in both themes, so the skill can never
teach something the validator refuses.

The hosted MCP server serves it as primer://skills/widgets (copied into dist
at build); the plugin's SKILL.md points live documents at widgets instead of
inline SVG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A checkpoint's bundle (C11) now holds each published widget as it looked,
labels included (widgets/<id>.light.svg and .dark.svg), plus its source and
rows (widgets/<id>.json); the static rendering and HTML export draw both
themes' SVG in place of the block, switched by the page's theme. Restoring a
checkpoint (C12) brings back the reachable sub-fragments it reaches (widget
labels now, page text and embeds as they land) and republishes each widget
from its kept source and rows when they render the kept hash.

Markdown export keeps a fenced `primer-widget` block naming the widget
instead of dropping it, and deleting a failed bundle removes every file its
manifest lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The widget ids in a Y.Doc are written by any editor, so a block could name
another document's widget and pull its source and rows into this document's
checkpoint, export or share, and a crafted bundle could overwrite that
widget on restore. Every lookup and update by a Y.Doc id is now scoped by the
document (and the widget's block); a block naming a foreign widget renders
"Widget unavailable". Y.Doc ids must be well formed (a UUID for a widget, a
plain name for a block) before they reach a query or a storage key, so a
traversal-shaped id is skipped, and restore applies a kept entry only when it
names the placed widget and block.

Restore also republishes widget rows before it writes the document, as a
publish does, so viewers refetching on the new hash get the restored SVG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fenced

The runtime check rewrote only url(#id) and left other url() values in
clip-path, mask, filter and marker attributes untouched, so a spread could
point one at a remote resource the host page would load. Any url() other than
url(#id) now refuses the render. The root font stack falls back to sans-serif
where Roboto isn't loaded, and the draft and publish tool descriptions say
that label words people wrote are untrusted data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the chain through M6 (M2b to M5, M10a with M9 records and M10d).
The widgets migration is regenerated as 0023 on top of 0022. Widget files in
a bundle are deleted with it and on purge (bundle keys come from the
manifest), the live print rendering draws widgets too, and the widget skill
is served by the same skill-resource loader as the authoring skill on both
MCP servers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… bound to records

Labels now use page text's fragment helpers (page/text.ts), so a label target
{ tabId, blockId, textId } is read, versioned, edited by ops `update`,
suggested (the suggestion keeps its textId, as a pid does) and commented like
page text; the widget's own label helpers are gone.

A thread on a widget part or data point is accepted only when the widget
block's live publish draws that part, or has that row and column; the
widget id comes from the client-written block, so the lookup is scoped by
document and block.

A draft can draw a records collection instead of inline rows: its shared
records become rows (id = record id), snapshotted so the publish is still
exactly the preview. A published widget stays bound, and each coalesced
data.changed redraws it from the new records, keeping the last good publish
when a redraw fails.

Agents read labels with their target and version, the words fenced as
untrusted, and an agent that may only suggest in a document doesn't draw
there, since a publish changes what people see at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d comments on parts and points

A widget block draws its live publish inline for the page's theme, through
the block services like a page block, and shows each label's current words
from the document, following every edit. Clicking a label opens the shared
overlay on its fragment: editors change the words, and anyone who may
comment comments on them (the thread store now scopes to a label as well as
page text). Clicking a mark drawn from a row comments on that data point,
and clicking a named part comments on the part; parts and points with open
threads are marked and listed under the widget.

The SVG is allowlisted by the server; it is checked again in the browser
before it enters Primer's own page, since it is not sandboxed there. The
page loads Roboto from the same package the server rasterises with, so the
agent's preview matches what people see.

The skills: authoring.md points at the widget skill, and widgets.md and the
reference cover records collections, label edits by ops and fenced labels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SVG was checked as XML and then injected as HTML, a parser differential
that CDATA, processing instructions, comments or a namespace trick could use
to slip markup past the check into Primer's own page. It is now parsed once
as XML, walked, and refused whole unless it holds only SVG-namespace
elements and text, with only the server's allowlisted tags and attributes,
no handlers, and references only inside the widget; the checked tree is
mounted with importNode. Hostile SVG (CDATA, a PI, a comment, an
HTML-namespace script, foreignObject, onload, xlink) shows as unavailable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rence

The url() check matched only the lowercase, unspaced form, so `URL(`,
`url (`, a CSS escape (`\75 rl(`) or a character reference (`&#117;rl(`)
could load an outside resource. On the server and in the browser, a value
with a backslash or a character reference is now refused, and `url(` in any
case or spacing passes only as exactly `url(#id)`. In the browser, paint
attributes must also be a colour, none or url(#id).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each comments listener opened its own EventSource, and each holds a
connection; with a widget or two on the page the browser's per-host
connection limit was reached and later requests (widget images) waited
forever. Listeners on a document now share one stream, closed with the last.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r and browser

The value check was a denylist, and CSS has more functions (image-set(),
src()) than any denylist keeps up with. api-types now holds one grammar per
attribute, used by the server's output check and by the browser's check
before a widget enters the page: reference attributes (mask, clip-path,
filter, markers) take only none or url(#id); paint takes a colour, none,
currentColor or url(#id); transform takes numbers only; any other attribute
holds no parenthesis, backslash or character reference. A table of hostile
values is refused by both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y value is capped

The transform pattern could backtrack catastrophically: its number form
overlapped itself and the separator was optional, so a long digit run
followed by a stray character hung the server's check and the browser's.
A transform is now read in one pass (function name, parenthesis, numbers in
one unambiguous form, parenthesis), grammar values are capped at 256
characters and every value at 100,000 before any matching. A 100k-digit
transform is refused at once by both checkers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The browser check showed widgets drawn at a third of the column (the block's
content box shrinks to fit) and the part comment box opening below the
viewport, where a fixed box can't be scrolled to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The changeset covers in-place labels, comments on parts and points, records
binding, checkpoints and the shared value grammar. The Vite config used for
the local browser check was committed by mistake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: widgets, agent-drawn diagrams and charts people edit and comment on (collaboration M8)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
const out = join(dir, `${WIDGET_FONT}-${face}.ttf`);
if (!existsSync(out)) {
const woff = readFileSync(require.resolve(`roboto-fontface/fonts/roboto/${WIDGET_FONT}-${face}.woff`));
writeFileSync(out, woffToSfnt(woff));
const out = join(dir, `${WIDGET_FONT}-${face}.ttf`);
if (!existsSync(out)) {
const woff = readFileSync(require.resolve(`roboto-fontface/fonts/roboto/${WIDGET_FONT}-${face}.woff`));
writeFileSync(out, woffToSfnt(woff));

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants