feat: add benchmark checker and scoring spec (task 0024)

.claude/tasks/0024-bench-checker-and-scoring-spec.md

@@ -12,8 +12,21 @@ This slice also defines the scoring-spec contract — a task's expected end stat
 
 ## Acceptance criteria
 
-- [ ] Given synthetic actual and expected state snapshots, the full-state diff passes when they match after normalization and fails when the actual state is missing the intended change.
-- [ ] The diff fails when the actual state carries collateral change beyond the intended mutation.
-- [ ] Normalization drops volatile identifiers and timestamps, matches comments by author and body, and compares label sets order-independently.
-- [ ] The read-task answer-match passes when the required facts are present in the final report and fails when a required fact is missing.
-- [ ] The scoring-spec contract expresses both a mutation's expected end state and a read's required answer facts.
+- [x] Given synthetic actual and expected state snapshots, the full-state diff passes when they match after normalization and fails when the actual state is missing the intended change.
+- [x] The diff fails when the actual state carries collateral change beyond the intended mutation.
+- [x] Normalization drops volatile identifiers and timestamps, matches comments by author and body, and compares label sets order-independently.
+- [x] The read-task answer-match passes when the required facts are present in the final report and fails when a required fact is missing.
+- [x] The scoring-spec contract expresses both a mutation's expected end state and a read's required answer facts.
+
+## Implementation Notes
+
+Two new files in `bench/`: `scoring-spec.ts` (the pure contract — `RepoState` and its `Label`/`Issue`/`PullRequest`/`Review`/`Comment` shapes, `RequiredFact`, and the `ScoringSpec` discriminated union) and `checker.ts` (the logic — `checkMutation`, `checkReadAnswer`, and a `score` entry point that dispatches on task kind). This mirrors the existing `result.ts` (shape) / `store.ts` (logic) split.
+
+Decisions and deviations worth flagging:
+
+- **Full-state diff covers the whole PR/issue surface, including reviews, assignees, and label definitions.** The criteria name only comments and label sets under normalization, but "diffing the entire post-run repository state" (User Story 5) and the contract's need to express the scored suite's review/merge/assignee tasks (criterion 5) make these part of the contract, not scope creep. They are groundwork the runner and task suite will populate.
+- **Reviews are matched order-independently**, consistent with how comments and labels are compared (spec line 9). A code review caught that reviews were initially order-dependent; this was fixed and covered by a test (`passes when a pull request's reviews match as a set despite differing order`). Review inline comments are likewise matched by author and body, order-independently.
+- **Failure diagnostics (`differences` naming the affected entity/label/comment/fact)** go beyond the bare pass/fail the criteria require, so a failed trial is traceable to what diverged (User Story 14 spirit). Heavily tested.
+- **Read-answer matching is deterministic substring matching** (case- and whitespace-normalized, with `anyOf` alternatives per fact), no LLM judge. This is intentionally naive — e.g. `"#42"` would match inside `"#420"` — and is mitigated by the task suite choosing disambiguating `anyOf` renderings rather than by the checker. The required facts carried in the `ScoringSpec` *are* the seeded ground truth for read tasks.
+- **`score` throws on a spec/submission kind mismatch** rather than silently scoring the wrong thing; this keeps the seam deterministic and is covered by a guard test.
+- No criteria were dropped; all five are satisfied.