feat: persist agent report on read result records (task 0032)
All checks were successful
CI / test (pull_request) Successful in 52s

Retain the agent's final report on the benchmark result record for read
tasks, so a failed read is diagnosable directly from the stored record
instead of only carrying an opaque `incorrect` tag. The runner resolves
the scoring spec once and records `run.finalReport` when the spec is a
read; mutation records omit the field entirely. The sample store needs no
change — it serializes whatever record it is handed.

This is the prerequisite for confirming the read-open-issue-count failure
from real report text before the state-aware count-line change (task 0033).
This commit is contained in:
2026-07-17 09:50:38 -04:00
parent 0fdc2bed9e
commit 4cbed21ff3
6 changed files with 258 additions and 6 deletions

View File

@@ -72,6 +72,15 @@ export interface ResultRecord {
/** Pass/fail outcome with a failure tag. */
outcome: Outcome;
/**
* The agent's final report, retained for read tasks so a failed read is
* diagnosable directly from the record — the exact text the agent submitted,
* distinguishing a wrong answer from a right one phrased in words the checker's
* accepted-phrasing list did not match. Absent for mutation tasks, which are
* scored by diffing repository state and have no agent report to record.
*/
report?: string;
}
/**