alexion/skills
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add axi-review skill

Browse Source 
Review a CLI against the 10 canonical AXI principles black-box: run a
staged battery of read and write probes against a user-provided target,
judge each principle strictly against axi.md, and emit a report-card
table.

Ships as SKILL.md plus AXI-PRINCIPLES.md (per-principle rubric with
probes and PASS/PARTIAL/FAIL criteria) and REPORT-FORMAT.md (the
single-table card format and verdict vocabulary).

Claude-Session: https://claude.ai/code/session_015uc7R6zNYZpCxFWJMXrxwh
This commit is contained in:
alexion
2026-07-22 23:22:27 -04:00
parent 49a73c30d1
commit 1caf18d72d
3 changed files with 260 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

141
skills/axi-review/AXI-PRINCIPLES.md Normal file
View File

@@ -0,0 +1,141 @@
# The 10 AXI Principles — review rubric
The canonical statements come from axi.md, which is the authority for the wording.
Each entry distils the canonical statement, gives the black-box probe to run, and the criteria that map observed output to a verdict.
Judge against the canonical statement, not against any single tool's interpretation.
Verdict states are defined in [`REPORT-FORMAT.md`](REPORT-FORMAT.md).
A deviation that the tool's own skill file documents is still judged by the canonical bar, with the documentation noted on the row.
## Efficiency
### Principle 1 — Token-efficient output
Canonical: structured output uses TOON (Token-Optimized Object Notation), not JSON, omitting braces, quotes, and commas while staying unambiguously parseable.
Probe: run a list command and a detail command, and inspect stdout.
- PASS: structured output is TOON — tabular `name[N]{f1,f2}:` blocks with no per-field JSON braces or quotes.
- PARTIAL: some commands emit TOON and others emit JSON or ad-hoc text.
- FAIL: the structured commands emit JSON or free-form prose.
Failure modes: emitting raw API JSON, pretty-printed JSON, or a human-oriented box-drawing table.
### Principle 2 — Minimal default schemas
Canonical: return 3–4 fields per list row by default, with more available through an explicit `--fields` flag.
Probe: run a list command with no `--fields` and count the fields per row, then confirm `--fields` adds fields.
- PASS: default list rows carry 3–4 fields and `--fields` opts into more.
- PARTIAL: the default carries 5–6 fields, or `--fields` is absent while the default stays lean.
- FAIL: the default carries 7 or more fields, or every field is always present with no minimal default.
Failure modes: returning the full API object by default, or offering no `--fields` mechanism.
### Principle 3 — Content truncation
Canonical: large text fields are truncated to a limit with a size hint appended, for example `(truncated, 2847 chars total — use --full to see complete body)`.
Probe: view an entity with a long body, then check that a list command carrying a body field truncates it too rather than leaking it raw, and confirm `--full` (or its equivalent) suppresses truncation on both.
The body-leak check belongs to the list command, not the detail view, where the body is always shown.
- PASS: long bodies truncate with an inline size hint that names how to get the full text, and `--full` returns the untruncated body.
- PARTIAL: it truncates without a hint, or hints without a `--full` escape, or truncates on detail while leaking raw bodies on a list path.
- FAIL: full bodies are emitted untruncated, or they truncate with no indication at all.
Failure modes: an unbounded body on a list path that spills many full bodies into context, or silent truncation with no hint.
## Robustness
### Principle 4 — Pre-computed aggregates
Canonical: include derived fields that eliminate round trips, always report total item counts, and provide inline summaries such as `27 passed, 0 failed, 10 skipped`.
Probe: run a list command and look for a leading count that reports a total, then run a command whose result has a natural rollup (status or checks) and look for an inline summary.
- PASS: list output leads with a total count such as `count: N of T total`, and derived summaries appear inline where relevant.
- PARTIAL: the count is bare (`count: N` with no total), or some aggregates are present while obvious round trips remain.
- FAIL: no counts and no derived fields, so the agent must issue follow-up calls for basics.
When the target has no data that exercises the rollup facet, for example no PRs or CI for a checks summary, judge on the count facet alone and note that the rollup was not observable.
Failure modes: a bare count with no total, or requiring a second call for a review decision or a status rollup.
### Principle 5 — Definitive empty states
Canonical: a query that returns nothing emits an explicit zero-result message rather than empty output, because an agent cannot tell a silent failure from a genuinely empty result.
Probe: run a query guaranteed to return nothing — a nonsense search or a no-match filter — and inspect stdout.
- PASS: empty results emit an explicit marker such as `<noun>[0]: (none)` or `0 open`, never silence.
- PARTIAL: some empty paths are marked while others print nothing, for example a dashboard that is blank when empty.
- FAIL: empty output on no results, indistinguishable from a silent failure.
Failure modes: printing nothing, or exiting 0 with blank stdout.
### Principle 6 — Structured errors and exit codes
Canonical: mutations are idempotent, errors are structured and written to stdout rather than stderr, commands never prompt for interactive input, exit code is 0 for success and 1 for errors and 2 for unknown flags, and unknown flags fail loudly rather than being silently ignored.
Probe, read facet (safe): trigger a not-found id, an unknown flag, and a bad target, then check for a structured error block on stdout with a code field and the right exit codes, with 2 for the unknown flag.
Probe, write facet: run a state-toggling mutation and then repeat it — one whose second run is a no-op, such as closing an already-closed entity or pinning an already-pinned one, not a create, which duplicates by design.
Observe an idempotent result on the repeat, an early return or an `already:` marker, and confirm no command prompts.
If the target rejects the write probes, the write facet is BLOCKED.
- PASS: errors are a structured block on stdout with a code, exit codes are 0/1/2 as canonical, unknown flags exit 2, no command prompts, and a repeated mutation is idempotent.
- PARTIAL: errors are structured but on stderr, or exit codes are collapsed to a single value, or an unknown flag is silently ignored, or some mutations are not idempotent.
- FAIL: errors are free-form text on stderr, or a command prompts interactively, or a repeated mutation duplicates its effect.
- BLOCKED: the write target rejected the mutation probes, so the idempotency facet could not be observed — record the reason.
Failure modes: writing errors to stderr, prompting for confirmation, or exiting 1 for every outcome.
## Discoverability
### Principle 7 — Ambient context
Canonical: install into the agent's session from an explicit setup command, so every conversation starts with relevant state visible, and ship an installable skill for on-demand guidance.
Probe: locate the tool's bundled Agent Skill file, under `~/.claude/skills/<tool>/` or in the installed package or repo, and check the surface for an explicit setup command.
- PASS: the tool ships a discoverable Agent Skill installed by an explicit setup command, optionally alongside a hook that injects a dashboard as initial context.
- PARTIAL: a setup command exists but installs no skill, or a skill exists with no explicit setup path, for example one installed by a postinstall script.
- FAIL: no discoverable skill file at all.
Failure modes: shipping no skill, or auto-installing via a postinstall script instead of an explicit setup command.
### Principle 8 — Content first
Canonical: running the command with no arguments displays live, actionable data rather than help text, preceded by the executable path (with `~` for home) and a one-sentence description of the tool's purpose.
Probe: run the bare invocation with no arguments.
- PASS: no-arg output is live tool state, preceded by a header naming the executable path and a one-sentence purpose.
- PARTIAL: it shows live data without the path-and-description header, or mixes help text with data.
- FAIL: no-arg output is a help or usage screen, or it errors.
Failure modes: printing usage text on no args, or showing no live data.
### Principle 9 — Contextual disclosure
Canonical: append `help[]` lines after output suggesting logical next steps as concrete command templates, keeping fixed disambiguating flags but leaving runtime values parameterized as placeholders like `<id>` rather than guessing them.
Probe: inspect the list and detail outputs for a `help[]` block, checking placeholders on list output and a filled id on unambiguous single-entity output.
- PASS: commands append a `help[]` block of concrete next-step templates, parameterizing runtime values as placeholders, and may fill a known id where a single entity makes it unambiguous.
- PARTIAL: suggestions appear on some commands but not others, or they hardcode a guessed runtime value, or the `help[]` block is empty.
- FAIL: no next-step suggestions anywhere.
Failure modes: guessing an id in a suggestion, or omitting suggestions.
### Principle 10 — Consistent way to get help
Canonical: each subcommand offers a concise `--help` flag as a fallback when contextual hints are insufficient.
Probe: run `--help` on the root and on at least one subcommand.
- PASS: every subcommand responds to `--help` with a concise flag reference.
- PARTIAL: `--help` works on some subcommands but not all, or its output is inconsistent or verbose.
- FAIL: no `--help`, or it errors or prompts.
Failure modes: `--help` missing on subcommands, or help that prompts interactively.

54
skills/axi-review/REPORT-FORMAT.md Normal file
View File

@@ -0,0 +1,54 @@
# Report card format
The card is rendered in-session as terminal markdown.
It reports all 10 principles, in canonical order grouped by category, so the whole rubric is visible rather than only the failures.
## Verdict states
- **PASS** — observed behaviour meets the canonical bar.
- **PARTIAL** — the principle is partly met, for example TOON output with no truncation hint.
- **FAIL** — observed behaviour violates the principle.
- **N/A** — the principle genuinely does not apply, for example an aggregate count on a tool with no list commands.
- **BLOCKED** — the probe could not be run, so behaviour was not observed. This is not a FAIL. It is reserved for the write facet of Principle 6 when the target rejects the mutation probes.
## Structure
Lead with a header naming the reviewed invocation and a one-line tally, then the skill-file status, then one table of all 10 principles in canonical order, then the artifacts manifest.
Each principle is one row.
The `Evidence` cell is the command that was run plus a short inline snippet of its output.
The `Fix / Note` cell carries a concrete fix when the verdict is not PASS, or the documented-deviation note when the skill file documents the deviation, and is left empty for a clean PASS.
```
# AXI review — <invocation>
**<n> PASS · <n> PARTIAL · <n> FAIL · <n> N/A · <n> BLOCKED**
Skill file: <path where found | MISSING — Principle 7 FAIL>
| # | Principle | Verdict | Evidence | Fix / Note |
|---|-----------|---------|----------|------------|
| 1 | Token-efficient output | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 2 | Minimal default schemas | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 3 | Content truncation | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 4 | Pre-computed aggregates | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 5 | Definitive empty states | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 6 | Structured errors and exit codes | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 7 | Ambient context | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 8 | Content first | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 9 | Contextual disclosure | <VERDICT> | `<command>` → <snippet> | <fix or note> |
| 10 | Consistent way to get help | <VERDICT> | `<command>` → <snippet> | <fix or note> |
## Artifacts created
<the exact artifacts the write battery created in the target, each with its identifier | none — writes were BLOCKED>
```
## Rules
Every non-PASS row carries a concrete fix in `Fix / Note`, a real remedy rather than a restatement of the principle.
The `Evidence` cell is a terse one-liner — the command plus a short inline snippet, never a multi-line dump.
When one command's output is the evidence for several principles, cite the same command in each row.
The artifacts manifest lists every artifact the write battery created, with the identifier needed to find it, so the user can remove anything they do not want.
When writes were BLOCKED and nothing was created, the manifest says so explicitly.

65
skills/axi-review/SKILL.md Normal file
View File

@@ -0,0 +1,65 @@
---
name: axi-review
description: Review a CLI for conformance to the 10 AXI (Agent eXperience Interface) principles by running it black-box and scoring each principle on a report card. Use when asked to review, audit, check, or measure a CLI's AXI conformance, or when "AXI principles", "Agent eXperience Interface", or axi.md is named.
---
# axi-review
Review a CLI against the 10 canonical AXI principles by running it and judging what it emits.
The review is black-box: the tool's observed output is the evidence, and source is consulted only where a facet cannot be triggered by running the tool.
It writes by default, so it creates real artifacts in a target you provide.
The rubric lives in [`AXI-PRINCIPLES.md`](AXI-PRINCIPLES.md) — one entry per principle, each with the probe to run and its PASS/PARTIAL/FAIL criteria.
The card's shape lives in [`REPORT-FORMAT.md`](REPORT-FORMAT.md).
## 1. Establish what you are reviewing
Fix the invocation string first — how this CLI is run (`gitea-axi`, a binary path, `npx -y foo`).
Locate the tool's own Agent Skill file, since it is the authoritative description of the command surface and is itself the evidence for Principle 7.
Look under `~/.claude/skills/<tool>/SKILL.md`, then in the installed package or the repo.
A missing skill file is a Principle 7 FAIL that you carry into the judging step.
When it is missing, enumerate the surface from `--help` and no-arg output instead.
Run the bare no-arg invocation to confirm the tool executes.
If nothing runs at all — binary absent, `npx` fails — stop and report that a behavioural review is impossible.
Emit no card in that case.
Done when the tool has run, you have its subcommand surface, and you know whether a skill file exists.
## 2. Get the write target
The write battery creates real artifacts, so it needs a target you are cleared to mutate — the repo, folder, or destination this CLI acts on.
How the tool takes that target comes from its skill file, for example gitea-axi's `-R owner/name`.
If the invoking prompt did not name a target, ask the user for one in prose before running any command.
Done when you have a write target you may create artifacts in.
## 3. Run the battery once, capturing a transcript
Read [`AXI-PRINCIPLES.md`](AXI-PRINCIPLES.md) and run every probe it lists, capturing each command's stdout and exit code into one transcript.
The battery is staged, not a single command: run the empty-state reads first, then the write probes, then the reads that depend on what the writes created.
Run it once in that sense — one transcript reused across principles, not a fresh run per principle — since one command's output is evidence for several.
The write probes include creating any fixture a read probe needs, such as the long-body entity Principle 3 inspects.
The read probes create nothing.
The write probes create real artifacts — record each one in the artifacts manifest as you create it, so nothing is orphaned silently.
Capture stdout and stderr separately, since Principle 6 turns on which stream an error lands in.
If the target rejects the write probes, mark the write facet of Principle 6 BLOCKED with the reason and continue with the read probes.
Done when every probe has a captured result or a recorded BLOCKED reason, and every created artifact is in the manifest.
## 4. Judge all 10 principles
For each principle in [`AXI-PRINCIPLES.md`](AXI-PRINCIPLES.md), apply its criteria to the transcript and assign PASS, PARTIAL, FAIL, N/A, or BLOCKED.
Judge strictly against the canonical statement.
When the tool's skill file documents a deviation, note it on the row but keep the canonical verdict.
Done when all 10 principles carry a verdict backed by a cited command and a short output excerpt from the transcript.
## 5. Emit the report card
Render the card in-session following [`REPORT-FORMAT.md`](REPORT-FORMAT.md).
Done when the card matches `REPORT-FORMAT.md` and every non-PASS row carries a concrete fix.