feat: persist the tool transcript on every result record
Records stored only a run's token/turn totals, so an arm's turn cost — the dominant driver of cache-read tokens — could not be diagnosed from the store. Retain the ordered transcript of tool invocations (the exact shell commands, MCP calls, and built-in tools the run made) on every scored record, absent only for a hung run that produced no transcript. The canonical TranscriptEntry shape lives on the record (result.ts); the isolation audit's ToolUse now aliases it so the persisted and audited shapes cannot drift.
This commit is contained in:
@@ -241,6 +241,12 @@ describe("runCell", () => {
|
||||
expect(sample.imputedCostUsd).toBe(DRIVER_COST);
|
||||
expect(sample.outcome).toEqual({ pass: true });
|
||||
|
||||
// The recorded sample carries the run's tool transcript — the exact ordered
|
||||
// sequence of tool invocations the driver reported — so the turn's cost is
|
||||
// diagnosable directly from the record. The expected value is the literal the
|
||||
// fake driver planted, deep-equal and in order, not recomputed from runner.ts.
|
||||
expect(sample.transcript).toEqual([{ kind: "mcp", server: "gitea-mcp", tool: "edit_issue" }]);
|
||||
|
||||
// The sample carries the run's wall-clock duration; a completed run takes
|
||||
// non-negative time.
|
||||
expect(typeof sample.durationMs).toBe("number");
|
||||
|
||||
Reference in New Issue
Block a user