The workflow pinned Node 20, which reached end-of-life in April 2026, while
the manifest promised support down to it — so the entire claimed range below
the single tested version went unverified and its floor was unsupported.
Matrix over the two supported long-term-support majors and narrow the engine
range to `^22 || ^24`, naming exactly what is tested. Narrowing is free now
because nothing has been published and no tags exist.
The benchmark harness tier joins every leg: it is deterministic and needs no
network, and its non-default runner configuration made it easy to believe it
was running when it was not. The end-to-end and packaging tiers run on the
highest leg only, conditioned on a flag attached to that leg through a matrix
`include` entry rather than on a version number restated at each site. The
benchmark smoke tier stays out, since it would pass by skipping.
`@types/node` follows the new floor; it was the last Node 20 reference in the
manifest, and the typecheck runs on every leg.
The Nix build now runs the installed-binary facet of the packaging tier
against the binary it has just produced, via an `installCheckPhase` that
sets `GITEA_AXI_INSTALLED_BIN` to `$out/bin/gitea-axi` and runs the new
`test:installed` script. Running after `fixupPhase` means the binary
under test is the wrapped one an operator actually gets, and naming it is
all the phase does — the assertions stay in the shared tier, so the npm
and Nix installation paths cannot drift apart in what they guarantee.
`npmInstallHook` prunes dev dependencies out of the build tree before the
check runs, so `preInstall` snapshots `node_modules` with `cp -al` and the
check restores it by copying, leaving the snapshot intact for a replayed
phase.
Two defects fixed in passing, both surfaced by review:
`test:installed` pins a test file by path while the packaging runner sets
`passWithNoTests: true`, so moving that file would have taken the build
green having asserted nothing — the same silently-inert trap `doCheck`
sprang in task 0037. The script now passes `--passWithNoTests=false`.
`checkPhase`'s vitest left a timestamped run cache under
`node_modules/.vite`, which `npmInstallHook` copied into `$out`, shipping
a stray cache and making the derivation non-reproducible. It is now
removed before the install phase; `nix build --rebuild` passes.
Add the maintainer-facing `bench:report` command, the offline counterpart
to `bench:run`: it opens the sample store, drains it, aggregates against the
scored suite and bonus definitions, and prints the aggregator's comparison to
stdout. It renders whatever has accumulated, annotating incomplete coverage
rather than blocking on a complete matrix.
`parseReportArgs` is the pure argument seam (`--store`, `--help`, plus a
`--self-review` / `--no-self-review` variant selector). Unlike `bench:run` the
boundary is offline — it reads only the local store, no host or Agent SDK — so
the whole `runReportCommand` is deterministic and unit-tested, not smoke-run.
Move `DEFAULT_STORE_ROOT` to `store.ts` as the single source of truth,
re-exported from `run.ts` for its existing importers.
Add the maintainer-facing command that runs one chosen benchmark cell on
demand, so only the token budget available at that moment is spent.
runCells (bench/run-loop.ts) runs one (arm, task) cell for a batch of
trials — defaulting to five with a reporting floor of three — by driving
the existing single-cell runner and the append-only sample store rather
than reimplementing orchestration. Re-running a cell deepens it: trial
numbering continues past the highest trial the cell already holds and the
new samples append, so a cell's sample size grows across sittings without
overwriting prior runs.
bench/run.ts is the command: parseRunArgs is the pure, unit-tested
argument seam, and runBenchCommand is the live boundary that resolves host
access, resolves the scored suite against the host's self-review support,
selects the task, and drives the run loop. It is invoked via the new
bench:run npm script, run under tsx (a new devDependency) because the
harness's .js-specifier imports need a TypeScript-aware runner. The Claude
Agent SDK is now declared as an optional peerDependency — documented but
neither installed for package consumers nor pulled into CI.
Every arm runs on the driver's single fixed model; the command exposes no
per-cell model override that could break cross-arm comparability. The
default store root bench/results/ is gitignored.
Add the deterministic, idempotent seed that brings a freshly provisioned
throwaway repository to a known ground truth before a trial runs, scripted
over the live Gitea API.
- bench/seed-plan.ts: the pure ground truth (fixed labels, an issue spread
across the discriminating dimensions, and labelled/reviewed/real-branch
pull requests) plus groundTruth(user), realizing it into a RepoState.
- bench/seed.ts: idempotent seeding reconciled by natural key, reusing
gitea-axi's own tea-login credential discovery (no new secret handling).
- A live smoke tier (test:bench:smoke) validating the seed end-to-end and
skipping cleanly when no host is configured, kept out of the deterministic
bench tier.
Export selectLogin from src/context.ts so the bench reuses the exact
credential-selection path.
Lay the foundation the benchmark harness reads and writes: a bench/
directory (excluded from the published npm package), the immutable
result-record shape, and an append-only per-cell sample store.
- bench/result.ts: the ResultRecord shape — four token components,
turns, duration, imputed cost, tagged pass/fail outcome, and the
arm/task/tier/trial/timestamp tags. Arm and Tier are typed unions.
- bench/store.ts: append-only sample store, one JSONL file per cell at
<root>/<arm>/<taskId>.jsonl; deepening a cell only ever adds samples.
- bench/README.md: harness working docs and benchmark vocabulary, kept
out of the tool's domain glossary per the spec.
- Dedicated bench test tier (vitest.bench.config.ts, npm run test:bench)
kept out of the fast tier; tsconfig typechecks bench.
- Packaging tier asserts bench/ never ships in the tarball.
Complete the distribution metadata and publish flow for the unscoped
`gitea-axi` package:
- add `repository`, `homepage`, and `bugs` to package.json
- add `publishConfig` (public access, npmjs registry) so `npm publish`
needs no extra flags
- replace `prepublishOnly` with `prepack: npm run build`, so both
`npm pack` and `npm publish` rebuild `dist/` first
- document the single-command flow in PUBLISHING.md
- add a packaging smoke-test tier (`test:pack`) that packs the real
tarball, installs it globally, and drives the installed binary
(`--help`, dashboard header, and `setup` finding the bundled skill)
Distribute gitea-axi's ambient context via explicit user actions (ADRs
0009, 0013), with no postinstall script:
- Bundle the Agent Skill markdown at skills/gitea-axi/SKILL.md (a
minimal pointer, not a command reference) and ship it via package.json
files.
- Add `setup`, which installs the skill into ~/.claude/skills/
idempotently (installed/updated/unchanged).
- Add `setup hooks`, which registers a SessionStart hook running the
bare dashboard for Claude Code, Codex, and OpenCode via the SDK's
installSessionStartHooks(), updating managed entries in place.
- Shadow the SDK's built-in `update` so it fails with VALIDATION_ERROR
and points at the npm update command, keeping the ten-code error list
intact.
Integration tests drive all three at the CLI seam against a temporary
HOME; these commands make no Gitea API calls, so there is no live-Gitea
e2e case.
Add v8 coverage via a test:coverage script and vitest config: measure src
only, exclude the types-only deps.ts and the bin entrypoint main.ts, and
gate on ratcheted thresholds (statements 85, branches 78, functions 90,
lines 85) set just under current coverage. CI runs the unit+integration
tier through coverage so a real drop fails the build.
Add the end-to-end test tier: a Gitea Actions workflow (GitHub-compatible)
that runs the unit+integration tiers plus e2e tests against a pinned
disposable Gitea service container. The e2e suite provisions its own repo,
token, and seed data over HTTP and drives the real CLI seam, gated on
GITEA_AXI_E2E_URL so the default suite stays dependency-free.
A fixture-vs-live shape guard anchors the issue-list FieldDef paths against
both the recorded fixture and the live response, failing on divergence.
Tracer bullet for gitea-axi: runnable npm package on axi-sdk-js with
gitea-js as the sole HTTP layer, ESM on Node 20+.
- issue list with --state/--limit, default fields, count line from
X-Total-Count, type=issues guard, explicit empty state, and next-step
suggestions
- repo context detection from the git origin remote (SSH/scp/HTTPS),
tea credential discovery with the three-way login-matching split, and
-R/--repo and --login overrides (flag > env > auto)
- token retrieval via tea login helper get: tea's login list JSON
carries no token (ADR 0001 amended)
- full AxiError classification table with path-based 404 split, TOON
errors on stdout, exit codes 0/1/2
- test mode (GITEA_AXI_API_URL/TOKEN/REPO) suppressing subprocesses,
fixture server, and vitest suites driving the CLI seam (50 tests)
