Spike: end-to-end tests for challenge skills #17

Why

#15 changed when the Mad Libs skill calls show_preview, and nothing could observe it. npm test covers hooks, servers, and extensions deterministically with no model calls, and a SKILL.md is prose the model follows. boothstrap dev --verify --live plays a package unattended but only reports done or missed per step; it never presses Open preview and doesn't record tool calls, so it would have passed before and after #15.

Question

What is the cheapest way to catch a skill regression (a step that stops reporting, a tool called at the wrong time, a question asked where the skill says not to) before it reaches a station?

Investigate

• Extending boothstrap's live driver to record MCP tool calls per step, with per-package expectations (for example: build-app calls show_preview before the agent's first question). Where would expectations live, and what does the driver need to expose?
• Reviewing live-run transcripts with a model against the skill's own rules, and how noisy that is.
• A nightly --live smoke run on a macOS runner with a Gemini key secret: what it catches (stalls, missed steps, broken hooks or resets, model or Pi drift), what it costs per night, and what it still can't see.
• Whether any of this can run on Linux, where this repo's CI lives.

Output

A short write-up in docs/research/YYYY-MM-DD-e2e-skill-tests.md with a recommendation and the follow-up issues to file, in this repo and in MLH/boothstrap.sh. No code beyond throwaway experiments.

Not part of the "Ship 3 challenges" milestone.