fix: run the bench agent in a neutral working directory
Some checks failed
CI / test (pull_request) Failing after 52s
Some checks failed
CI / test (pull_request) Failing after 52s
The SDK driver ran the agent with no explicit cwd, so its shell inherited the harness's own checkout. When the agent omitted `-R OWNER/NAME`, the gitea-axi (and tea) CLI defaulted the repository from that local checkout — silently resolving the harness repo instead of the seeded throwaway — and returned a plausible but wrong result (e.g. `count: 0 open of 0 total` for a repo with no issues). This contaminated read-tier scoring for the checkout- defaulting arms and was surfaced by the newly persisted read reports. Give each run a fresh, empty working directory outside any checkout, so a forgotten `-R` errors instead of hitting the wrong repository, and delete it when the run ends.
This commit is contained in:
@@ -1,5 +1,7 @@
|
||||
import { readdirSync, rmSync, statSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { sumTokens } from "./sdk-driver.js";
|
||||
import { createAgentWorkdir, sumTokens } from "./sdk-driver.js";
|
||||
import type { SdkResultMessage } from "./sdk-driver.js";
|
||||
|
||||
describe("sumTokens", () => {
|
||||
@@ -87,3 +89,31 @@ describe("sumTokens", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("createAgentWorkdir", () => {
|
||||
// Behavior: each agent run must operate in a fresh, empty directory located
|
||||
// OUTSIDE the harness's own checkout. This isolation is what stops an agent
|
||||
// that forgets an explicit `-R OWNER/NAME` from having its `gitea-axi`/`tea`
|
||||
// tools silently default their target repo to the harness's own git checkout:
|
||||
// a directory that is empty (no `.git`) and outside the current checkout gives
|
||||
// those tools nothing local to resolve.
|
||||
//
|
||||
// The three assertions are independent anti-bug properties drawn directly from
|
||||
// the requirement, not recomputed from the implementation:
|
||||
// 1. the path exists and is a directory,
|
||||
// 2. it is empty (zero entries — in particular no `.git`),
|
||||
// 3. it sits outside the current working directory (relative path escapes
|
||||
// upward with "..").
|
||||
it("returns a fresh, empty directory located outside the current checkout", () => {
|
||||
const dir = createAgentWorkdir();
|
||||
try {
|
||||
expect(statSync(dir).isDirectory()).toBe(true);
|
||||
expect(readdirSync(dir)).toHaveLength(0);
|
||||
expect(path.relative(process.cwd(), dir).startsWith("..")).toBe(true);
|
||||
} finally {
|
||||
// Safe: `dir` is a fresh throwaway temp dir we just received from
|
||||
// createAgentWorkdir(); never a delete of cwd or any pre-existing path.
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user