docs: sync core-parser spec and add unit normalisation
Bring core-parser.md in line with the resolved design in CONTEXT.md and ADRs 0004/0005, and promote a new decision to normalise ingredient/cookware units. - Full sync of the core-parser spec: diagnostics/total-function channel, container blocks (ListItemBlock, BlockquoteBlock), RawInline alongside RawBlock via position-slicing, the unified WikilinkNode/TransclusionNode shape as two typed nodes, and the minimal remark plugin set. - New ADR 0006: normalise ingredient/cookware units via a known-alias table with passthrough, case-insensitive, canonical-only; align Timer matching to case-insensitive too. - SPEC.md: units are normalised (new Units section, alias table); Timer table marked case-insensitive. - CONTEXT.md: add Unit Normalisation entry, resolve the open item. - fixtures/basic.md: add a blockquote (callout) carrying an annotation and a non-canonical unit so the integration test exercises containers and normalisation. - Expand the core test.todo checklists to cover the new behaviour (containers, raw fallbacks, diagnostics, unit normalisation, wikilink/transclusion variants); still pending, suite stays green.
This commit is contained in:
@@ -6,47 +6,69 @@ Tooling that needs to extract structured data from Recipe Files — the CLI, and
|
||||
|
||||
## Solution
|
||||
|
||||
Implement the Parser in `@kitchen-md/core`: a pure function that accepts a Recipe File string and returns a Document AST.
|
||||
The Parser uses remark internally to handle CommonMark and OFM syntax, runs a custom annotation transform to surface Ingredient, Cookware, and Timer nodes, and exposes a clean public API using core's own types — no remark internals leak through.
|
||||
Implement the Parser in `@kitchen-md/core`: a single pure, total function that accepts a Recipe File string and returns a Document AST.
|
||||
It never throws — every input produces a usable Document AST, and the one class of genuinely invalid input (malformed frontmatter YAML) is reported through a diagnostics channel rather than as an exception (see ADR 0004).
|
||||
Internally the Parser uses remark with a deliberately minimal plugin set to handle CommonMark and OFM, runs a custom annotation transform to surface Ingredient, Cookware, and Timer nodes, and exposes a clean public API built entirely from core's own types — no remark internals leak through (see ADR 0003).
|
||||
|
||||
## User Stories
|
||||
|
||||
1. As a tooling user, I want to call `parse` with a Recipe File string and receive a Document AST so that I can build structured tooling without writing a parser myself.
|
||||
2. As a tooling user, I want frontmatter passed through as a plain object so that I can read whatever fields I need without the parser imposing a schema.
|
||||
3. As a tooling user, I want heading blocks in the Document AST so that I can render or reason about document structure.
|
||||
4. As a tooling user, I want paragraph blocks with typed inline nodes so that I can render prose and distinguish plain text from structured content.
|
||||
5. As a tooling user, I want list blocks with ordered and unordered variants so that recipe steps written as lists are represented faithfully.
|
||||
6. As a tooling user, I want code blocks passed through without annotation parsing so that example syntax in documentation is not mistakenly extracted as recipe data.
|
||||
7. As a tooling user, I want wikilinks represented as typed Inline Nodes so that I can render or process `[[links]]` distinctly from plain text.
|
||||
8. As a tooling user, I want transclusions — including KitchenMD Step References — represented as typed Inline Nodes so that tooling can resolve them.
|
||||
9. As a tooling user, I want Ingredient Annotations represented as Inline Nodes carrying name, quantity, and unit so that I can extract ingredient data from any recipe prose.
|
||||
10. As a tooling user, I want Cookware Annotations represented as Inline Nodes carrying name and optional quantity so that I can extract equipment requirements.
|
||||
11. As a tooling user, I want Timer Annotations represented as Inline Nodes carrying a value or range and a canonical unit so that I can extract timing information.
|
||||
12. As a tooling user, I want annotations inside code spans and code blocks to be ignored so that the parser follows CommonMark inline processing rules.
|
||||
13. As a future Obsidian plugin author, I want to import `@kitchen-md/core` without taking a transitive dependency on remark so that my plugin bundle does not include remark's internals.
|
||||
3. As a tooling user, I want `parse` to never throw and to report malformed frontmatter as a diagnostic so that I never have to wrap it in `try`/`catch`, yet can still distinguish a broken frontmatter block from an absent one.
|
||||
4. As a tooling user, I want heading blocks in the Document AST so that I can render or reason about document structure.
|
||||
5. As a tooling user, I want paragraph blocks with typed inline nodes so that I can render prose and distinguish plain text from structured content.
|
||||
6. As a tooling user, I want list blocks with ordered and unordered variants, whose items are containers holding blocks, so that recipe steps written as lists are represented faithfully and annotations inside them are still extracted.
|
||||
7. As a tooling user, I want blockquote blocks modelled as containers so that prose and annotations inside blockquotes and callouts are preserved and extracted rather than lost.
|
||||
8. As a tooling user, I want code blocks passed through without annotation parsing so that example syntax in documentation is not mistakenly extracted as recipe data.
|
||||
9. As a tooling user, I want any OFM construct the parser does not model preserved verbatim as a raw block or raw inline so that no source is ever discarded and a document round-trips unchanged.
|
||||
10. As a tooling user, I want wikilinks represented as typed Inline Nodes so that I can render or process `[[links]]` distinctly from plain text.
|
||||
11. As a tooling user, I want transclusions — including KitchenMD Step References — represented as typed Inline Nodes so that tooling can resolve them.
|
||||
12. As a tooling user, I want Ingredient Annotations represented as Inline Nodes carrying name, quantity, and unit so that I can extract ingredient data from any recipe prose.
|
||||
13. As a tooling user, I want Cookware Annotations represented as Inline Nodes carrying name, quantity, and unit so that I can extract equipment requirements.
|
||||
14. As a tooling user, I want Timer Annotations represented as Inline Nodes carrying a value or range and a canonical unit so that I can extract timing information.
|
||||
15. As a tooling user, I want ingredient and cookware units matching a known alias normalised to a canonical abbreviation, with unknown units passed through unchanged, so that I get consistent units for aggregation without losing unusual free-form ones.
|
||||
16. As a tooling user, I want annotations inside code spans and code blocks to be ignored so that the parser follows CommonMark inline processing rules.
|
||||
17. As a future Obsidian plugin author, I want to import `@kitchen-md/core` without taking a transitive dependency on remark so that my plugin bundle does not include remark's internals.
|
||||
|
||||
## Implementation Decisions
|
||||
|
||||
- The public API exports a single `parse` function.
|
||||
Given a Recipe File string, it returns a `DocumentAST`.
|
||||
It is a pure function — no filesystem access, no side effects.
|
||||
It is a pure function — no filesystem access, no side effects — and a total function that never throws (see ADR 0004).
|
||||
|
||||
- `DocumentAST` top-level shape: `{ frontmatter, blocks }`.
|
||||
- `DocumentAST` top-level shape: `{ frontmatter, blocks, diagnostics }`.
|
||||
- `frontmatter` — the raw parsed YAML metadata as a plain object, passed through without schema enforcement.
|
||||
Empty object when no frontmatter is present.
|
||||
Empty object when no frontmatter is present, and also empty when the frontmatter block is present but invalid.
|
||||
- `blocks` — a flat, ordered array of Block nodes representing the document body.
|
||||
The flat structure means headings are siblings of their following content, not parents of it.
|
||||
The array is flat at the top level: headings are siblings of their following content, not parents of it.
|
||||
Container blocks (list items, blockquotes) nest their own child blocks; the flatness applies to the top level, not inside containers.
|
||||
See ADR 0003 and the Document AST entry in the domain glossary.
|
||||
- `diagnostics` — an ordered array of non-fatal Diagnostic warnings surfaced during parsing.
|
||||
Empty in the normal case.
|
||||
|
||||
- **Total function and diagnostics** (see ADR 0004):
|
||||
- `parse` never throws.
|
||||
Genuinely invalid input is reported through `diagnostics`, not exceptions, so a consumer always receives a usable Document AST.
|
||||
- The only diagnostic emitted today is `invalid-frontmatter`: when the `---` block is present but not valid YAML, the body still parses, `frontmatter` is `{}`, and the raw YAML plus the parse error are preserved in the diagnostic.
|
||||
- A `Diagnostic` has the shape `{ severity, code, message, source?, position? }`.
|
||||
`severity` is a level such as `"warning"`; `code` is a stable machine-readable identifier (e.g. `"invalid-frontmatter"`); `message` is human-readable; `source` preserves the offending raw text verbatim; `position` locates it in the input.
|
||||
`Diagnostic` is a public API type; adding new codes later is additive and non-breaking.
|
||||
|
||||
- **Block types** (initial set):
|
||||
- `HeadingBlock` — level (1–6) and an inline content array.
|
||||
- `ParagraphBlock` — an inline content array.
|
||||
- `ListBlock` — ordered flag and an array of list items, each carrying an inline content array.
|
||||
- `ListBlock` — an ordered flag and an array of `ListItemBlock`.
|
||||
- `ListItemBlock` — a container block holding a `Block[]`.
|
||||
A list item wraps its own child blocks (typically a paragraph), not a bare inline array, so `- a $ladle{}` becomes a list item containing a paragraph.
|
||||
This mirrors how CommonMark models list items and is what lets annotation extraction reach inside them.
|
||||
- `BlockquoteBlock` — a container block holding a `Block[]`.
|
||||
OFM callouts (`> [!note]`) carry no dedicated node; they parse as ordinary blockquotes whose text is preserved, and annotations inside them are extracted like any other container content.
|
||||
- `CodeBlock` — optional language identifier and a literal text string.
|
||||
Annotation parsing does not run on code block content.
|
||||
- `ThematicBreakBlock` — no content fields.
|
||||
- `RawBlock` — carries the raw text of any remark node type not explicitly modelled above.
|
||||
Ensures forward compatibility when the fixture or future files use block types not yet in the catalogue.
|
||||
- `RawBlock` — the block-level raw fallback for any remark node type not explicitly modelled above (see ADR 0005).
|
||||
It exposes a single `value` string and nothing else — no `kind` or type hint.
|
||||
`value` is captured by position-slicing the original input, so it is byte-for-byte what the author wrote and round-trips unchanged.
|
||||
|
||||
- **Inline Node types**:
|
||||
- `TextNode` — a literal string of plain text.
|
||||
@@ -55,29 +77,68 @@ The Parser uses remark internally to handle CommonMark and OFM syntax, runs a cu
|
||||
- `CodeSpanNode` — a literal string.
|
||||
Annotation parsing does not run on code span content.
|
||||
- `LinkNode` — href string and an inline content array.
|
||||
- `WikilinkNode` — target (filename without extension) and optional display text.
|
||||
Covers both `[[link]]` and `[[link|display]]`.
|
||||
- `TransclusionNode` — target and optional anchor string.
|
||||
Covers `![[link]]`, `![[link#heading]]`, and KitchenMD Step References (`![[file#section:N]]` and `![[file#N]]`).
|
||||
The anchor is passed through as-is; Step Reference resolution is out of scope for the parser.
|
||||
- `WikilinkNode` — `{ target, anchor?, display? }`.
|
||||
`target` is the filename without extension; `anchor` is the part after `#`, passed through verbatim; `display` is the alias after `|`.
|
||||
Covers `[[link]]`, `[[link#heading]]`, and `[[link|display]]`.
|
||||
- `TransclusionNode` — the same `{ target, anchor?, display? }` shape as `WikilinkNode`, but a distinct node type marking an embed (`![[…]]`).
|
||||
The node type — not a boolean flag — is the discriminator between a reference and an embed.
|
||||
Covers `![[link]]`, `![[link#heading]]`, `![[link|alt]]`, and KitchenMD Step References (`![[file#section:N]]` and `![[file#N]]`), whose anchor is passed through as-is; Step Reference resolution is out of scope for the parser.
|
||||
- `IngredientNode` — name, optional quantity string, optional unit string.
|
||||
- `CookwareNode` — name, optional quantity string, optional unit string.
|
||||
- `TimerNode` — value string (single) or value + high strings (range), and a canonical unit abbreviation.
|
||||
- `RawInline` — the inline-level raw fallback for any remark inline node type not explicitly modelled above (see ADR 0005).
|
||||
Like `RawBlock`, it exposes only a position-sliced verbatim `value` string and no `kind` hint.
|
||||
|
||||
- remark is used internally as the CommonMark parser (see ADR 0002).
|
||||
OFM wikilinks and transclusions are handled by `remark-wiki-link` and its underlying micromark and mdast-util packages.
|
||||
KitchenMD annotations are processed by a custom remark transform plugin that walks mdast text nodes outside code contexts and splits them into annotation Inline Nodes.
|
||||
remark types do not appear in `@kitchen-md/core`'s public API (see ADR 0003).
|
||||
An internal translation layer maps remark's mdast output to core's own types before returning.
|
||||
- The internal remark plugin set is deliberately minimal (see ADR 0002 and ADR 0005): `remark-parse` + `remark-frontmatter` + `remark-gfm` (tables, task lists, strikethrough, autolinks) + `remark-wiki-link` (links and embeds), plus core's own annotation transform.
|
||||
Most other OFM syntax needs no dedicated plugin: `==highlight==`, `%%comment%%`, `$math$`, and `$$block math$$` are left as literal text with their markers intact, and callouts parse as ordinary blockquotes.
|
||||
Anything remark tokenises into a node core does not model (e.g. a GFM table, strikethrough) falls through to `RawBlock` or `RawInline`.
|
||||
remark types do not appear in `@kitchen-md/core`'s public API; an internal translation layer maps remark's mdast output to core's own types before returning, recursing into container children rather than collapsing them.
|
||||
|
||||
- Annotation parsing naturally respects CommonMark inline scoping: the annotation transform plugin operates only on mdast text nodes, which do not appear inside code spans or code blocks.
|
||||
No additional scoping logic is required.
|
||||
- KitchenMD annotations are processed by a custom remark transform plugin that walks mdast text nodes outside code contexts and splits them into annotation Inline Nodes.
|
||||
Annotation parsing naturally respects CommonMark inline scoping: the transform operates only on mdast text nodes, which do not appear inside code spans, code blocks, HTML comments, or raw HTML blocks, so no additional scoping logic is required.
|
||||
Because it only rewrites text nodes (which are always modelled, never raw), it does not invalidate the parse positions the raw fallbacks depend on.
|
||||
|
||||
- Timer unit aliases are normalised to canonical abbreviations at parse time (`mins` → `min`, `hours` → `hr`, etc.).
|
||||
The full alias table is defined in the Timer Annotation entry of the domain glossary.
|
||||
- The Ingredient/Cookware quantity/unit split is grammar-driven, not last-space-driven.
|
||||
The quantity is matched greedily against the numeric grammar (mixed number → fraction → decimal → integer) anchored at the start of the braces, and any non-empty remainder after the delimiting space is the unit.
|
||||
If the brace content does not begin with a grammar-valid quantity (e.g. `@stock{a splash}`), the entire content is preserved as the quantity string and the unit is left empty, so no annotation is dropped.
|
||||
Quantity strings are preserved as-is (e.g. `"1 1/2"`, `"0.5"`, `"1/2"`); no numeric coercion or arithmetic is performed at the parser level.
|
||||
|
||||
- Quantity strings are preserved as-is (e.g. `"1 1/2"`, `"0.5"`, `"1/2"`).
|
||||
No numeric coercion or arithmetic is performed at the parser level.
|
||||
- **Ingredient and Cookware unit normalisation** (see ADR 0006):
|
||||
- A unit whose text matches a known alias is normalised to its canonical abbreviation; any unit not in the table is passed through verbatim (original casing and spacing preserved).
|
||||
- Lookup is case-insensitive; the node stores only the canonical unit — the author's original spelling is not retained.
|
||||
- Multi-word aliases match on the whole unit remainder (e.g. `200 fluid ounces` → unit `fl oz`).
|
||||
- Alias table (case-insensitive; anything unlisted passes through unchanged):
|
||||
|
||||
| Canonical | Aliases |
|
||||
|-----------|---------|
|
||||
| `g` | g, gram, grams |
|
||||
| `kg` | kg, kilogram, kilograms, kilo, kilos |
|
||||
| `mg` | mg, milligram, milligrams |
|
||||
| `oz` | oz, ounce, ounces |
|
||||
| `lb` | lb, lbs, pound, pounds |
|
||||
| `ml` | ml, milliliter, millilitre, milliliters, millilitres |
|
||||
| `l` | l, liter, litre, liters, litres |
|
||||
| `tsp` | tsp, teaspoon, teaspoons |
|
||||
| `tbsp` | tbsp, tablespoon, tablespoons |
|
||||
| `cup` | cup, cups |
|
||||
| `fl oz` | fl oz, fluid ounce, fluid ounces |
|
||||
| `pt` | pt, pint, pints |
|
||||
| `qt` | qt, quart, quarts |
|
||||
| `gal` | gal, gallon, gallons |
|
||||
|
||||
- Single-letter cooking abbreviations (`t`, `T`, `c`) are intentionally excluded — they are ambiguous and would collide under case-insensitive lookup.
|
||||
Count and descriptive units (`clove`, `pinch`, `dash`, `can`, `large`, `to taste`) have no canonical form and pass through unchanged.
|
||||
|
||||
- **Timer unit normalisation**: aliases are normalised to canonical abbreviations at parse time, using the same case-insensitive matching as ingredient/cookware units (see ADR 0006).
|
||||
Alias table (case-insensitive):
|
||||
|
||||
| Canonical | Aliases |
|
||||
|-----------|---------|
|
||||
| `s` | sec, secs, second, seconds |
|
||||
| `min` | min, mins, minute, minutes |
|
||||
| `hr` | hr, hrs, hour, hours |
|
||||
|
||||
The unit matches only as a complete word and the longest known alias wins; `~5 minsx` is not a timer because `minsx` is not a known unit.
|
||||
|
||||
- Core's own types are defined in a dedicated types module within `@kitchen-md/core` and re-exported from the package entry point alongside `parse`.
|
||||
|
||||
@@ -90,17 +151,24 @@ The Parser uses remark internally to handle CommonMark and OFM syntax, runs a cu
|
||||
Each test passes a Recipe File string to `parse` and asserts the returned Document AST.
|
||||
Coverage must include:
|
||||
- Frontmatter passthrough (arbitrary fields, empty frontmatter, no frontmatter)
|
||||
- Each Block type: heading (all levels), paragraph, ordered list, unordered list, code block, thematic break
|
||||
- Each Inline Node type: plain text, emphasis, strong, code span, link, WikilinkNode, TransclusionNode
|
||||
- Malformed frontmatter — `parse` does not throw, `frontmatter` is `{}`, body still parses, and an `invalid-frontmatter` diagnostic is emitted with the raw YAML preserved
|
||||
- Each Block type: heading (all levels), paragraph, ordered list, unordered list, code block, thematic break, blockquote
|
||||
- `ListItemBlock` wraps a paragraph (a list item holds child blocks, not a bare inline array)
|
||||
- Annotation extraction inside a container (an annotation inside a blockquote or list item is surfaced)
|
||||
- Each Inline Node type: plain text, emphasis, strong, code span, link, `WikilinkNode`, `TransclusionNode`
|
||||
- `WikilinkNode` with an anchor and with a display alias; `TransclusionNode` with a display alias
|
||||
- A remark node core does not model falls through to `RawBlock` (block level, e.g. a GFM table) and `RawInline` (inline level), with the verbatim source preserved
|
||||
- Ingredient Annotation: name, quantity, unit; multi-word name; unit-less quantity; no quantity
|
||||
- Cookware Annotation: name with quantity and unit; name with no quantity; multi-word name
|
||||
- Timer Annotation: single value form; range form; all supported unit aliases normalised to canonical form
|
||||
- Ingredient/Cookware unit normalisation: a known alias normalised to canonical (`grams` → `g`, `Tbsp` → `tbsp`), case-insensitive matching, a multi-word unit (`fluid ounces` → `fl oz`), and an unknown unit passed through verbatim
|
||||
- Timer unit matching is case-insensitive (`~5 Mins` → `min`)
|
||||
- Annotations embedded mid-sentence (not at the start of a line)
|
||||
- Annotations inside code spans — not extracted
|
||||
- Annotations inside code blocks — not extracted
|
||||
- A Step Reference transclusion — anchor passed through as-is
|
||||
|
||||
- **Integration tests** pass the content of `fixtures/basic.md` through `parse` and assert the complete Document AST, covering all annotation types, OFM features, and frontmatter together in a single realistic input.
|
||||
- **Integration tests** pass the content of `fixtures/basic.md` through `parse` and assert the complete Document AST, covering all annotation types, OFM features (including a blockquote with an annotation and a non-canonical unit that normalises), and frontmatter together in a single realistic input.
|
||||
|
||||
- Tests are co-located with the source module and follow the `{module}_test.ts` naming convention established in the project scaffold.
|
||||
|
||||
@@ -108,8 +176,6 @@ The Parser uses remark internally to handle CommonMark and OFM syntax, runs a cu
|
||||
|
||||
- Quantity arithmetic or numeric normalisation (e.g. `1/2` → `0.5`).
|
||||
Quantities are preserved as strings.
|
||||
- Unit normalisation for Ingredient and Cookware units.
|
||||
Those units are free-form and passed through as-is.
|
||||
- Step counting and Step Reference resolution.
|
||||
The parser surfaces `TransclusionNode` with the raw anchor; resolution is a consumer concern.
|
||||
- Combined Recipe validation.
|
||||
@@ -122,5 +188,6 @@ The Parser uses remark internally to handle CommonMark and OFM syntax, runs a cu
|
||||
- The "built from scratch" statement in the project spec refers to not forking Cooklang's parser.
|
||||
Using remark as an internal dependency is consistent with that intent — see ADR 0002.
|
||||
- The annotation transform plugin running only on text nodes is what enforces CommonMark scoping for free, without any explicit code-span or code-block detection logic in the plugin itself.
|
||||
- `RawBlock` is a safety valve, not a target.
|
||||
Implementation should aim to model all block types that appear in recipe fixtures explicitly.
|
||||
- `RawBlock` and `RawInline` are safety valves, not targets.
|
||||
The implementation models every construct that appears in recipe fixtures explicitly and lets only genuinely unmodelled syntax fall through to a raw node.
|
||||
Their verbatim `value` is captured by position-slicing the original input — never re-stringified — so unmodelled constructs round-trip byte-for-byte.
|
||||
|
||||
Reference in New Issue
Block a user