feat: persist agent report on read result records (task 0032)
All checks were successful
CI / test (pull_request) Successful in 52s

Retain the agent's final report on the benchmark result record for read
tasks, so a failed read is diagnosable directly from the stored record
instead of only carrying an opaque `incorrect` tag. The runner resolves
the scoring spec once and records `run.finalReport` when the spec is a
read; mutation records omit the field entirely. The sample store needs no
change — it serializes whatever record it is handed.

This is the prerequisite for confirming the read-open-issue-count failure
from real report text before the state-aware count-line change (task 0033).
This commit is contained in:
2026-07-17 09:50:38 -04:00
parent 0fdc2bed9e
commit 4cbed21ff3
6 changed files with 258 additions and 6 deletions

View File

@@ -102,4 +102,24 @@ describe("SampleStore", () => {
secondRun.append(second);
expect(secondRun.read(cell)).toEqual([first, second]);
});
// Behavior: the store round-trips a report-bearing record without any change to
// the store itself. A read task's record carries the agent's final report in the
// `report` field; appending it and reading it back must return it equal, `report`
// and all, proving the store persists the field with no store change. The planted
// report string is an independent literal, not anything production code computes.
it("round-trips a record carrying a report field, preserving it unchanged", () => {
const store = createSampleStore(root);
const report = "There are 5 open issues in the repository.";
const record = sample({ tier: "read", report });
store.append(record);
const readBack = store.read({ arm: record.arm, taskId: record.taskId });
expect(readBack).toEqual([record]);
const [only] = readBack;
expect(only).toBeDefined();
if (only === undefined) return;
expect(only.report).toBe(report);
});
});