[lacraig2]

A test runner driven by per-test metadata, plus the test machines it uses as fixtures. This is the base for selecting tests by what a change touches. This PR only adds the runner. run.sh, guest.yml and ci.yml are unchanged.

Test machines

tests/testmachine.py turns a tests/mkbusybox.py image into a machine for tests that run guest programs:

• How it's built. The image's kernel and machine arguments, with an initramfs containing the image's busybox and a hypercall agent (tests/programs/agent.c) as /init.
• Snapshot. The machine is snapshotted at the agent's first poll. That costs one boot (3–6 s), and the snapshot is cached next to the image.
• Using it. A test reverts to the snapshot in about 0.5 s, then puts files, runs commands and gets files through hypercalls and guest memory. There is no boot and no serial console.
• Arches: x86_64, aarch64 and arm (busybox-armel). tests/README.md lists what another arch needs: an image, the system layer in programs/sys.h, the agent build and the machine entry.

Metadata and tests/run

Each test file declares a literal PANDA_TEST = dict(...). Its fields:

Field	Meaning
area, tier, arches, fixture, plugins	what the test covers, and what it needs
variants	a variant set, with a default per dimension and per-arch restriction
xfail	expected failures, strict
always	the test always runs
args/env	values with placeholders
needs	other prerequisites
timeout	per-test limit
nix_check	the flake check that runs this script

tests/pandatest, a pytest plugin, reads the metadata with ast. Collecting tests therefore imports neither pandare2 nor an engine. It makes one item per test, arch and variant, with an id such as testmachine[x86_64,intel=1,smp=4].

tests/run:

• Selection. Picks items by area, name, tier and arch, and explains why each item is selected or skipped.
• Arch names. Engine target names; mkbusybox's names are accepted as aliases.
• Isolation. Each item runs in its own process, pinned, with its own work directory and TMPDIR.
• Scheduling.
  • Static and harness items run in parallel.
  • Guest items run one at a time, each behind a resource guard: free disk, available memory, and no build running.
  • A failed guard waits up to $PANDATEST_GUARD_WAIT, then skips the item with the reason.
• Variants. --variants default runs each variant dimension at its default only.
• Results.
  • JUnit XML, plus a JSON record per item: outcome, reason, duration, log and fixture identity.
  • Each record also notes the engine build's tree and revision, and whether it had uncommitted changes.
  • An xfail that passes is reported as XPASS and fails the run.

Fixtures are named:

• busybox-<arch>
• testmachine-<arch>[-smpN][-intel]
• cloud-<distro>-<arch>

Each name resolves to a cached path with an identity. A missing fixture skips its items unless --build-fixtures is given.

Metadata is added for:

• the static tier;
• the construct and multi harnesses;
• smoke;
• the busybox taint tests;
• testmachine.py.

The new flake check flake-checks (tests/static/flake_checks.py) checks that every flake check that runs a test script is the nix_check its metadata names. The dev shell's Python gains pytest.

Testing

Against nix build .#python, with fixtures built by --build-fixtures from tests/busybox.sh images.

tests/run --arch x86_64,aarch64: 29 pass, 48 skip, each with its reason.

Passed	Items
Static	13
construct, multi	2
smoke	2 arches
taint, taint_mmio, taint_errors	2 arches each
testmachine	6 variants: x86_64 smp 1/4 × intel 0/1, aarch64 smp 1/4

Skipped	Why
Tests without metadata yet	not run by tests/run
7 static tests	need the engine source tree, which a nix build doesn't expose

The other checks:

Check	Result
tests/busybox.sh (12 arches)	36/36 pass
tests/smoke.py x86_64	passes
nix flake check	passes, including flake-checks
examples/consumer flake check	passes