Skip to content

Run the evals with a Node outside the home directory - #30

Merged
kevinccbsg merged 1 commit into
mainfrom
evals/node-outside-home
Oct 26, 2026
Merged

kevinccbsg merged 1 commit into
mainfrom
evals/node-outside-home

Conversation

@kevinccbsg

Copy link
Copy Markdown
Member

The eval sandbox cannot read the home directory, so with Node from nvm npx fails in every run (Cannot find module …/bin/npx). With a Node from the official installer, npx twd-cli works: a twd-run-loop run scored 1.00 with no extra grants, and for the first time ran the failing test alone before fixing it.

  • The Quick and Full commands put /usr/local/bin first in PATH and drop the two grants for calling the twd-cli stand-in directly.
  • The note explains why npx twd-lint and npx tsc still only get attempted: the sandbox has no network, so they cannot fetch their package. Those graders check the call, the way a test of code that calls an external service does.
  • The baseline table stays until the next full run, and now says it was measured with the extra grants.

The sandbox cannot read the home directory, so npx from nvm fails in
every run. A Node from the official installer or apt works, which drops
the grants twd-run-loop needed to reach the twd-cli stand-in.
@kevinccbsg
kevinccbsg merged commit 2418bba into main Oct 26, 2026
1 check passed
@kevinccbsg
kevinccbsg deleted the evals/node-outside-home branch October 26, 2026 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant