alexion
9bf8c85dc3
Lay the foundation the benchmark harness reads and writes: a bench/ directory (excluded from the published npm package), the immutable result-record shape, and an append-only per-cell sample store. - bench/result.ts: the ResultRecord shape — four token components, turns, duration, imputed cost, tagged pass/fail outcome, and the arm/task/tier/trial/timestamp tags. Arm and Tier are typed unions. - bench/store.ts: append-only sample store, one JSONL file per cell at <root>/<arm>/<taskId>.jsonl; deepening a cell only ever adds samples. - bench/README.md: harness working docs and benchmark vocabulary, kept out of the tool's domain glossary per the spec. - Dedicated bench test tier (vitest.bench.config.ts, npm run test:bench) kept out of the fast tier; tsconfig typechecks bench. - Packaging tier asserts bench/ never ships in the tarball.
14 lines
561 B
TypeScript
14 lines
561 B
TypeScript
import { defineConfig } from "vitest/config";
|
|
|
|
export default defineConfig({
|
|
test: {
|
|
// The benchmark harness tier. The harness lives in bench/ (excluded from the
|
|
// published npm package) and its deterministic seams — the sample store,
|
|
// checker, guard, and aggregator — are unit-tested here, colocated with the
|
|
// source. It runs on its own via `test:bench` and is kept out of the main
|
|
// fast tier so bench code never counts against src coverage thresholds.
|
|
include: ["bench/**/*.test.ts"],
|
|
passWithNoTests: true,
|
|
},
|
|
});
|