fix: pre-authenticate the gitea-axi bench arm via its env interface
The gitea-axi arm was the only shell arm handed no credentials: the runner set only PATH, so the agent had to reverse-engineer the tea-login system — guessing a profile name and hunting for a config file — before any real work, burning ~4 turns per task. Since turns drive cache-read, the benchmark's dominant cost metric, this scaffolding gap alone inflated gitea-axi's cost-equivalent tokens above every other arm. Hand the arm its host and token through gitea-axi's own env interface (GITEA_AXI_API_URL / GITEA_AXI_TOKEN), the symmetric counterpart to the gitea-mcp server's GITEA_HOST / GITEA_ACCESS_TOKEN env: both name the same two facts, and both still leave the agent to name the repository per call. A shell arm now carries a credential env (empty for tea and raw-api, which need none), merged under PATH in the driver. Also strengthen SKILL.md so a cold agent targets and authenticates on the first call: an explicit "Targeting and authentication" section replaces the buried, optional-looking one-liner, spelling out that outside a checkout `-R OWNER/NAME` plus the environment's token is all that is needed — do not go hunting for a config file or login profile. Verified live: create-memory-leak-issue dropped from 10 turns to 3 and its cache-read fell ~3.8x, with the auth flailing gone from the transcript.
This commit is contained in:
@@ -271,7 +271,10 @@ function buildOptions(
|
||||
if (arm.shell !== null) {
|
||||
// Lead the agent's PATH with the arm's curated bin directory so only its one
|
||||
// allowed binary resolves by name; the guard on canUseTool is the authority.
|
||||
options.env = { ...process.env, PATH: arm.shell.path };
|
||||
// Layer the arm's credential env underneath so its tool is pre-authenticated
|
||||
// the way its product is really configured, symmetric to the gitea-mcp
|
||||
// server's env (see ArmShell.env); PATH stays last so it is never overridden.
|
||||
options.env = { ...process.env, ...arm.shell.env, PATH: arm.shell.path };
|
||||
}
|
||||
if (arm.mcp !== null) {
|
||||
options.mcpServers = { [arm.arm]: { type: "stdio", ...arm.mcp.server } };
|
||||
|
||||
Reference in New Issue
Block a user