test(nix): drive the installed binary through the shared tier (task 0038) #47

alexion · 2026-07-19T23:23:33-04:00

alexion commented

2026-07-19 23:23:33 -04:00

Implements .claude/tasks/0038-nix-build-drives-installed-binary.md.

What was built

The Nix build now drives the binary it has just installed through the shared installed-binary tier, so nix build and npm run test:pack guarantee the same things about an installed gitea-axi.

The check lives in installCheckPhase, which runs after fixupPhase — so the binary under test is the wrapped one an operator actually gets, not the bare entrypoint. It sets GITEA_AXI_INSTALLED_BIN=$out/bin/gitea-axi and runs npm run test:installed, a new script running the installed-binary facet alone. Naming the binary is all the phase does; the assertions stay in the tier.

npmInstallHook prunes dev dependencies out of the build tree during installPhase, so vitest is gone by the time the check runs. preInstall snapshots node_modules with cp -al (hardlinks — free, and the prune's deletions don't follow them) and the check restores it by copying, so a replayed phase doesn't consume the only snapshot.

vitest.packaging.config.ts joined the source allowlist — exactly the Gotcha task 0037 recorded, hit on the first build.

Both negative criteria were confirmed by deliberate sabotage probes: stripping the executable bit fails all three tests with EACCES, and moving the installed skills directory aside fails the setup test. nix build --rebuild now passes too, so the output is bit-for-bit reproducible.

Deviations

Criterion 2 holds, but by a different mechanism than the task assumed. The task motivates the executable-bit guard with "the compiler does not set the executable bit that npm would otherwise set from the manifest's bin entry". That's true of the npm path and not of the Nix path: nodejsInstallExecutables installs each bin entry as a generated wrapper invoking node <path>, not as a symlink. So chmod -x dist/main.js changes nothing (verified — the build stayed green), while chmod -x $out/bin/gitea-axi fails. The criterion as written is satisfied and was demonstrated, but the bit it protects under Nix is one makeWrapper always sets. That's an argument for the shared tier rather than against it: neither path picks which guarantees it feels like offering.

Two defects fixed in passing, both surfaced by review rather than planned:

test:installed pins a test file by path while the packaging runner sets passWithNoTests: true — moving that file would have taken nix build green having asserted nothing, the same silently-inert trap doCheck sprang in task 0037. Now passes --passWithNoTests=false.
checkPhase's vitest left a timestamped run cache under node_modules/.vite, which npmInstallHook copied into $out — shipping a stray cache and making the derivation non-reproducible. Strictly a 0037 defect, but this task adds a second vitest run over the same surface and the fix is one line in a file already being edited.

Scope note: .gitignore gains result / result-*, which is 0037's flake output rather than this task's.

Review

Risk

Overall: LOW

Blast radius: Low — confined to the Nix derivation's install phases plus a new npm script alias; no runtime source touched.
Reversibility: Low — pure additions; reverting is a clean delete with no state, schema, or published-API residue.
Test coverage: Low — the change is test wiring, validated positively (build, fast tier, nix flake check) and negatively (two sabotage probes).
Sensitive domain: Low — no auth, permissions, payments, concurrency, or migrations.
Size & complexity: Low — ~25 lines of linear shell across two phases, documented inline.
Runtime criticality: Low — build-time only; a failure breaks nix build loudly rather than shipping a defect.

Unaddressed findings

Standards — test:pack and test:installed share the vitest run --config vitest.packaging.config.ts prefix (Duplicated Code, judgement call). Left alone: the reviewer itself put it below the extraction threshold, and npm scripts have no good extraction mechanism short of a wrapper script that would obscure both.

Spec — criterion 2 is arguably vacuous under Nix and "should be re-scoped or marked as covered by test:pack rather than by nix build". Left as [x] rather than [-]: the criterion as literally written was demonstrated by probe. The nuance is recorded in the task's Implementation Notes, in a package.nix comment, and as a CLAUDE.md Gotcha, which seemed better than silently rewriting an acceptance criterion after the fact.

Everything else the review raised is fixed in the diff.

Implements [`.claude/tasks/0038-nix-build-drives-installed-binary.md`](.claude/tasks/0038-nix-build-drives-installed-binary.md). ## What was built The Nix build now drives the binary it has just installed through the shared installed-binary tier, so `nix build` and `npm run test:pack` guarantee the same things about an installed gitea-axi. The check lives in `installCheckPhase`, which runs after `fixupPhase` — so the binary under test is the *wrapped* one an operator actually gets, not the bare entrypoint. It sets `GITEA_AXI_INSTALLED_BIN=$out/bin/gitea-axi` and runs `npm run test:installed`, a new script running the installed-binary facet alone. Naming the binary is all the phase does; the assertions stay in the tier. `npmInstallHook` prunes dev dependencies out of the build tree during `installPhase`, so vitest is gone by the time the check runs. `preInstall` snapshots `node_modules` with `cp -al` (hardlinks — free, and the prune's deletions don't follow them) and the check restores it by copying, so a replayed phase doesn't consume the only snapshot. `vitest.packaging.config.ts` joined the source allowlist — exactly the Gotcha task 0037 recorded, hit on the first build. Both negative criteria were confirmed by deliberate sabotage probes: stripping the executable bit fails all three tests with `EACCES`, and moving the installed `skills` directory aside fails the `setup` test. `nix build --rebuild` now passes too, so the output is bit-for-bit reproducible. ## Deviations **Criterion 2 holds, but by a different mechanism than the task assumed.** The task motivates the executable-bit guard with "the compiler does not set the executable bit that npm would otherwise set from the manifest's `bin` entry". That's true of the npm path and *not* of the Nix path: `nodejsInstallExecutables` installs each `bin` entry as a generated wrapper invoking `node <path>`, not as a symlink. So `chmod -x dist/main.js` changes nothing (verified — the build stayed green), while `chmod -x $out/bin/gitea-axi` fails. The criterion as written is satisfied and was demonstrated, but the bit it protects under Nix is one `makeWrapper` always sets. That's an argument for the shared tier rather than against it: neither path picks which guarantees it feels like offering. **Two defects fixed in passing**, both surfaced by review rather than planned: - `test:installed` pins a test file by path while the packaging runner sets `passWithNoTests: true` — moving that file would have taken `nix build` green having asserted nothing, the same silently-inert trap `doCheck` sprang in task 0037. Now passes `--passWithNoTests=false`. - `checkPhase`'s vitest left a timestamped run cache under `node_modules/.vite`, which `npmInstallHook` copied into `$out` — shipping a stray cache and making the derivation non-reproducible. Strictly a 0037 defect, but this task adds a second vitest run over the same surface and the fix is one line in a file already being edited. **Scope note:** `.gitignore` gains `result` / `result-*`, which is 0037's flake output rather than this task's. ## Review ### Risk **Overall: LOW** - Blast radius: Low — confined to the Nix derivation's install phases plus a new npm script alias; no runtime source touched. - Reversibility: Low — pure additions; reverting is a clean delete with no state, schema, or published-API residue. - Test coverage: Low — the change *is* test wiring, validated positively (build, fast tier, `nix flake check`) and negatively (two sabotage probes). - Sensitive domain: Low — no auth, permissions, payments, concurrency, or migrations. - Size & complexity: Low — ~25 lines of linear shell across two phases, documented inline. - Runtime criticality: Low — build-time only; a failure breaks `nix build` loudly rather than shipping a defect. ### Unaddressed findings **Standards — `test:pack` and `test:installed` share the `vitest run --config vitest.packaging.config.ts` prefix (Duplicated Code, judgement call).** Left alone: the reviewer itself put it below the extraction threshold, and npm scripts have no good extraction mechanism short of a wrapper script that would obscure both. **Spec — criterion 2 is arguably vacuous under Nix and "should be re-scoped or marked as covered by `test:pack` rather than by `nix build`".** Left as `[x]` rather than `[-]`: the criterion as literally written was demonstrated by probe. The nuance is recorded in the task's Implementation Notes, in a `package.nix` comment, and as a CLAUDE.md Gotcha, which seemed better than silently rewriting an acceptance criterion after the fact. Everything else the review raised is fixed in the diff.

alexion added 1 commit 2026-07-19 23:23:34 -04:00

test(nix): drive the installed binary through the shared tier (task 0038)

CI / test (pull_request) Successful in 53s

Details

CI / test (push) Successful in 54s

Details

82cdad4ce0

The Nix build now runs the installed-binary facet of the packaging tier
against the binary it has just produced, via an `installCheckPhase` that
sets `GITEA_AXI_INSTALLED_BIN` to `$out/bin/gitea-axi` and runs the new
`test:installed` script. Running after `fixupPhase` means the binary
under test is the wrapped one an operator actually gets, and naming it is
all the phase does — the assertions stay in the shared tier, so the npm
and Nix installation paths cannot drift apart in what they guarantee.

`npmInstallHook` prunes dev dependencies out of the build tree before the
check runs, so `preInstall` snapshots `node_modules` with `cp -al` and the
check restores it by copying, leaving the snapshot intact for a replayed
phase.

Two defects fixed in passing, both surfaced by review:

`test:installed` pins a test file by path while the packaging runner sets
`passWithNoTests: true`, so moving that file would have taken the build
green having asserted nothing — the same silently-inert trap `doCheck`
sprang in task 0037. The script now passes `--passWithNoTests=false`.

`checkPhase`'s vitest left a timestamped run cache under
`node_modules/.vite`, which `npmInstallHook` copied into `$out`, shipping
a stray cache and making the derivation non-reproducible. It is now
removed before the install phase; `nix build --rebuild` passes.

alexion merged commit 82cdad4ce0 into main

2026-07-19 23:27:31 -04:00

alexion deleted branch task-0038-nix-build-drives-installed-binary

2026-07-19 23:27:31 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: alexion/gitea-axi#47