alexion/gitea-axi
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add benchmark checker and scoring spec (task 0024)
All checks were successful
CI / test (pull_request) Successful in 51s
Details
CI / test (push) Successful in 48s
Details

Browse Source
This commit was merged in pull request #25.
This commit is contained in:
alexion
2026-07-16 07:33:31 -04:00
parent 0436dc25fd
commit 66f576a2b7
6 changed files with 873 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
bench/README.md
View File

@@ -34,6 +34,8 @@ The raw component breakdown is retained on every sample so the data can be re-we
- `result.ts` — the immutable result-record shape and its tags (arm, task, tier, trial, timestamp).
- `store.ts` — the append-only, per-cell sample store that accumulates result records.
- `guard.ts` — the authoritative tool-isolation guard plus the curated per-arm bin directory that backs it.
- `scoring-spec.ts` — the scoring-spec contract: a task's expected end state (mutation) or required answer facts (read), consumed by the checker and produced by the runner and task suite.
- `checker.ts` — the deterministic scorer: the full-state diff for mutation tasks and the answer-match for read tasks, plus the `score` entry point that dispatches on task kind.
Later slices add the seed provisioning, the arm scaffolding, the single-cell runner, the task suite, the run-loop CLI, and the aggregator.