skills/.claude/tasks/0006-trend-history-and-ribbons.md

spec, blocked-by

spec	blocked-by
skill-benchmarking	0005-regression-arm-and-two-verdicts

What to build

The longitudinal layer that lets a real change be told apart from run-to-run noise, plus the fragility flags that stop a barely-green skill from masquerading as robust.

History. The deterministic core appends one summary line per run to a per-skill history file and trims it, oldest-first, at roughly the last fifty. Each line records both the efficacy and the regression net-margin (wins minus losses) and pass/fail, plus a flag for whether the run was two-arm or three-arm, so both ribbons can plot their own sparkline and correctly show a gap for any two-arm run. The history is local and ephemeral: it lives inside the git-ignored reports directory and resets if that directory is cleaned, consistent with reports being transient artifacts.

Trend ribbons. The report grows two stacked trend ribbons — an Efficacy ribbon and a Regression ribbon — each a net-margin sparkline over roughly the last seven runs, with verdict-colored per-run dots, the current run ringed, and a readout of the current net, the change versus the previous run, and how many recent runs were green. The Regression ribbon's dot is absent for any run that was two-arm. The Regression series is comparable across runs only while main is unchanged; because the previous-version arm is always main's HEAD, merging the branch moves main and resets the meaningful regression history, which is accepted rather than normalized.

Fragility chips and the clean resting state. A clean run rests fully collapsed, and fragility is flagged per badge. A green Efficacy badge carries a "narrowest margin" chip on the case fewest trial-flips from failing efficacy — closest to dropping under three wins. A green Regression badge independently carries a chip on the case sitting at exactly one loss. A case fragile on both axes shows both chips, so a barely-green skill cannot look as safe as a clean sweep on either axis.

The core's fixture unit test is extended to cover the history append-and-trim, the two ribbons' rendering including the two-arm gap in the regression series, and the per-badge fragility chips.

Acceptance criteria

• The core appends one summary line per run to a per-skill history file and trims oldest-first at roughly fifty lines.
• Each history line records the efficacy and regression net-margins and pass/fail plus a two-arm/three-arm flag.
• The history file lives inside the git-ignored reports directory and is treated as ephemeral (resets when that directory is cleaned).
• The report renders two stacked ribbons (Efficacy and Regression), each a net-margin sparkline over ~7 runs with verdict-colored dots, the current run ringed, and a net/delta/green-count readout.
• The Regression ribbon shows no dot for any run that was two-arm.
• A clean run rests fully collapsed.
• A green Efficacy badge carries a narrowest-margin chip on the case fewest trial-flips from failing efficacy; a green Regression badge independently carries a chip on the case at exactly one loss; a both-axes-fragile case shows both.
• The core's fixture unit test covers the history append-and-trim, both ribbons (including the two-arm gap), and the per-badge fragility chips.

Implementation Notes

The whole slice lives in the deterministic core, core/benchmark_core.py, and is fully covered by the extended checks/benchmark-core.nix fixture test. The runner's SKILL.md step 6 gains the --history tests/.reports/<name>.history.jsonl argument and a description of the append-trim and the ribbons, since the orchestration half runs in-session and is established by running the harness rather than unit-tested.

• The standard /implement branching flow was followed this time. Task 0005's PR had merged to main, so the blocker was reachable: main was fast-forwarded to origin/main and task-0006 branched off it, without the manual stack-tip fast-forward 0005 needed.
• History persistence is opt-in via --history and is the core's only side effect. Without the flag the core stays a pure transform and renders no ribbons, so the 0005-era report shape is unchanged. The history is JSON-lines — one run summary object per line — read, appended, trimmed to fifty, and rewritten.
• The ribbon delta compares against the previous run that carries a value on that axis, not the literal previous line. On the regression axis this skips a two-arm gap rather than blanking the change readout, consistent with the spec accepting regression discontinuity across a main move.
• The green-count readout shows green/applicable, not a bare green count. The denominator is the number of runs in the window that have a value on that axis, so a regression readout excludes two-arm runs it cannot score, which reads more honestly than counting them against the total.
• The sparkline draws a faint dashed zero baseline when the plotted window straddles zero, as a readability aid for telling a positive net-margin run from a negative one.
• Two committed fixtures were added, machine-generated for accuracy with the generator not committed (the 0005 precedent). The existing red fixtures carry no green axis and so exercise no chips, so clean-bundle.json (fully clean, green/green) pins the fully-collapsed resting state and the lone narrowest-margin efficacy chip, and fragile-bundle.json (green but fragile) pins the independent per-badge chips and the both-axes case carrying both.
• The efficacy chip is a single case, the regression chip is every case at exactly one loss. This asymmetry follows the wording: "narrowest margin" is comparative and picks one case (ties broken by authored order), while "the case sitting at exactly one loss" is an absolute condition any number of cases can meet.
• A case at exactly one regression loss both auto-expands and carries its chip. "A clean run rests fully collapsed" holds because a clean run has zero losses, so nothing trips the 0005 flagged-loss auto-expand — the chip is what still flags fragility on a case that has no loss to open it (a green efficacy case sitting at exactly three wins).