docs: resolve core parser design decisions

Capture the design decisions reached while grilling the core-parser spec:

- Parser is a total function; invalid frontmatter YAML surfaces through a
  new Document AST `diagnostics` channel instead of throwing (ADR 0004).
- Parse all of OFM losslessly via `RawBlock`/`RawInline` fallbacks with
  position-sliced verbatim source and a minimal plugin set (ADR 0005).
- Grammar-driven (not last-space) quantity/unit split, fixing the
  mixed-number collision, with trimming and multi-word unit rules.
- Container blocks nest `Block[]` so annotations surface inside
  blockquotes and list items; unified wikilink/transclusion shape.

Update SPEC.md and CONTEXT.md accordingly; no parser code yet.
This commit is contained in:
2026-07-14 20:52:15 -04:00
parent 66bb497aca
commit 3ffd2b3f9f
4 changed files with 178 additions and 13 deletions

View File

@@ -21,7 +21,11 @@ The `{}` wrapper is mandatory — bare `@name` without braces is not valid.
Quantity is optional; `@black pepper{}` is valid with no quantity. Quantity is optional; `@black pepper{}` is valid with no quantity.
Valid quantity formats: integer (`2`), decimal (`0.5`), simple fraction (`1/2`), mixed number (`1 1/2`). Valid quantity formats: integer (`2`), decimal (`0.5`), simple fraction (`1/2`), mixed number (`1 1/2`).
Unit is optional and free-form — the format places no constraints on what unit string is written. Unit is optional and free-form — the format places no constraints on what unit string is written.
When both quantity and unit are present, they are delimited by the last space inside the braces — so `@butter{1 1/2 tbsp}` parses as quantity `1 1/2`, unit `tbsp`. The quantity/unit split is **grammar-driven, not last-space-driven**: the parser matches the quantity greedily against the numeric grammar (mixed number → fraction → decimal → integer) anchored at the start of the braces, and any non-empty remainder after the delimiting space is the unit.
`@butter{1 1/2 tbsp}` → quantity `1 1/2`, unit `tbsp`; `@butter{1 1/2}` → quantity `1 1/2`, no unit (a plain last-space rule would wrongly split this into `1` / `1/2`, which is why the grammar is authoritative).
If the brace content does not begin with a grammar-valid quantity (e.g. `@stock{a splash}`), the entire content is preserved as the quantity string and the unit is left empty — no annotation is dropped.
The name is trimmed of surrounding whitespace (interior spaces preserved, so `@unsalted butter{}` stays `unsalted butter`), and the brace content is trimmed before grammar-matching.
The unit is the whole remainder after the quantity, so multi-word units work: `@stock{200 fl oz}` → quantity `200`, unit `fl oz`.
Valid name characters: any character except `{`, `}`, `@`, `$`, `~`. Valid name characters: any character except `{`, `}`, `@`, `$`, `~`.
## Cookware Annotation ## Cookware Annotation
@@ -31,7 +35,7 @@ Syntax: `$name{}`, `$name{quantity}`, or `$name{quantity unit}`
Same `{`-terminated name rule as Ingredient. Same `{`-terminated name rule as Ingredient.
The `{}` wrapper is mandatory. The `{}` wrapper is mandatory.
Quantity is optional; `$pan{}` is valid with no quantity. Quantity is optional; `$pan{}` is valid with no quantity.
Quantity and unit follow the same rules as Ingredient: valid formats are integer, decimal, simple fraction, mixed number; unit is free-form; delimited by the last space inside the braces. Quantity and unit follow the same rules as Ingredient: valid formats are integer, decimal, simple fraction, mixed number; unit is free-form; the split is grammar-driven, with non-numeric content preserved wholesale as the quantity string.
Valid name characters: any character except `{`, `}`, `@`, `$`, `~`. Valid name characters: any character except `{`, `}`, `@`, `$`, `~`.
## Timer Annotation ## Timer Annotation
@@ -43,7 +47,10 @@ The `~` sigil is followed immediately by a number or `NN` range, a space, the
N follows the same format rules as ingredient/cookware quantity: integer, decimal, simple fraction, or mixed number. N follows the same format rules as ingredient/cookware quantity: integer, decimal, simple fraction, or mixed number.
Supports natural range syntax: `~10-15 mins`. Supports natural range syntax: `~10-15 mins`.
In range form, the `-` separator follows the first quantity expression — since negative quantities don't exist, the `-` is unambiguous. `~1/2-1 hr` parses as a range of 1/2 hr to 1 hr. In range form, the `-` separator follows the first quantity expression — since negative quantities don't exist, the `-` is unambiguous. `~1/2-1 hr` parses as a range of 1/2 hr to 1 hr.
The range `-` must directly abut both numbers: `~10-15 mins` is a range, but `~10 - 15 mins` (spaces around `-`) is plain prose, not a timer.
One or more spaces separate the number (or range) from the unit.
The annotation terminates after the unit word — trailing prose is ignored. `~1 min more` parses as a timer of 1 min; "more" is plain text. The annotation terminates after the unit word — trailing prose is ignored. `~1 min more` parses as a timer of 1 min; "more" is plain text.
The unit matches only as a complete word (followed by whitespace, punctuation, or end-of-input) and the longest known alias wins (`seconds` before `sec`); `~5 minsx` is not a timer because `minsx` is not a known unit.
Known unit set (aliases → canonical abbreviation): `sec`, `secs`, `second`, `seconds``s`; `min`, `mins`, `minute`, `minutes``min`; `hr`, `hrs`, `hour`, `hours``hr`. Known unit set (aliases → canonical abbreviation): `sec`, `secs`, `second`, `seconds``s`; `min`, `mins`, `minute`, `minutes``min`; `hr`, `hrs`, `hour`, `hours``hr`.
## Recipe File ## Recipe File
@@ -109,28 +116,71 @@ See **Document AST** for the exact shape.
## Document AST ## Document AST
The full structured representation of a Recipe File returned by the Parser. The full structured representation of a Recipe File returned by the Parser.
Top-level shape: `{ frontmatter: Record<string, unknown>, blocks: Block[] }`. Top-level shape: `{ frontmatter: Record<string, unknown>, blocks: Block[], diagnostics: Diagnostic[] }`.
`frontmatter` is the raw YAML metadata, passed through without schema enforcement. `frontmatter` is the raw YAML metadata, passed through without schema enforcement (empty object when absent or invalid).
`blocks` is a flat, ordered list of Block nodes representing the document body in document order. `blocks` is a flat, ordered list of Block nodes representing the document body in document order.
The flat structure means headings and their following content are siblings, not parent/child. The flat structure means headings and their following content are siblings, not parent/child.
`diagnostics` is a list of non-fatal [[Diagnostic]] warnings surfaced during parsing (empty in the normal case).
The Parser is a **total function**: it never throws. Genuinely invalid input (currently only malformed frontmatter YAML) is reported through `diagnostics`, not exceptions, so a consumer always receives a usable Document AST — mirroring how Obsidian still renders a note whose frontmatter is broken. See ADR 0004.
Consumers that need section grouping derive it by scanning for Heading nodes. Consumers that need section grouping derive it by scanning for Heading nodes.
Core defines its own AST node types; remark (the internal Markdown parser) is a private implementation detail and its types do not appear in the public API. Core defines its own AST node types; remark (the internal Markdown parser) is a private implementation detail and its types do not appear in the public API.
See ADR 0002 and ADR 0003. See ADR 0002, ADR 0003, ADR 0004, and ADR 0005.
## Diagnostic
A non-fatal warning surfaced by the Parser through the Document AST's `diagnostics` array, instead of throwing.
Shape: `{ severity, code, message, source?, position? }`.
`severity` is a level such as `"warning"`; `code` is a stable machine-readable identifier (e.g. `"invalid-frontmatter"`); `message` is human-readable; `source` preserves the offending raw text verbatim (e.g. the invalid YAML); `position` locates it in the input.
The only diagnostic emitted today is `invalid-frontmatter`: when the `---` block is present but not valid YAML, the body still parses, `frontmatter` is `{}`, and the raw YAML plus the parse error are preserved in a diagnostic for the consumer to render as a warning.
Diagnostics are the extensible channel for any future non-fatal parse issue.
See ADR 0004.
## Block ## Block
A top-level node in the Document AST's `blocks` array. A node in the Document AST's `blocks` array.
Each Block represents one logical unit of document structure: a heading, a paragraph, a list, a code block, etc. Each Block represents one logical unit of document structure: a heading, a paragraph, a list, a code block, a blockquote, etc.
Block types are defined by core; the exact catalogue is not yet finalised — see open questions below. Block types are defined by core.
Some blocks are **container blocks** that nest other blocks rather than holding inline content directly: a blockquote holds `Block[]`, and a list item is itself a block container holding `Block[]` (so `- a $ladle{}` becomes a list item wrapping a paragraph, not a bare inline array).
This mirrors how CommonMark actually models these constructs and is what lets annotation parsing reach inside blockquotes and list items, as the language spec's annotation scope requires.
Annotation extraction runs over the internal Markdown tree's text nodes everywhere outside code, so annotations surface in any container automatically; the translation layer recurses into container children instead of collapsing them to a [[Raw Block / Raw Inline|Raw Block]].
## Inline Node ## Inline Node
A node representing inline content within a Block (e.g. within a paragraph or list item). A node representing inline content within a Block (e.g. within a paragraph or list item).
Inline nodes include: plain text, emphasis, strong, code span, link, Wikilink, and the three Annotation types (Ingredient, Cookware, Timer). Inline nodes include: plain text, emphasis, strong, code span, link, Wikilink, and the three Annotation types (Ingredient, Cookware, Timer).
## Wikilink / Transclusion
The two OFM link Inline Nodes.
A **Wikilink** (`[[…]]`) is a reference; a **Transclusion** (`![[…]]`) is an embed.
Their inner syntax is identical in OFM — they differ only by the leading `!` — so they share one structural shape: `{ target, anchor?, display? }`.
`target` is the referenced filename without the `.md` extension.
`anchor` is the part after `#`, passed through verbatim: a heading (`#Batter`), a block reference (`#^abc`), or a KitchenMD [[Step Reference]] (`#section:N` or `#N`).
`display` is the alias after `|` (`[[page|shown]]`, `![[page|alt]]`).
Anchor resolution — including Step Reference resolution — is a consumer concern, not the Parser's.
## Lossless OFM Parsing
The Parser must parse all of Obsidian Flavored Markdown without ever failing or discarding source.
The required internal remark plugin set is deliberately minimal: `remark-parse` + `remark-frontmatter` + `remark-gfm` (OFM's tables, task lists, strikethrough, autolinks) + `remark-wiki-link` (links and embeds), plus core's own annotation transform.
That set is sufficient because most non-CommonMark OFM syntax is already lossless with no plugin — remark leaves `==highlight==`, `%%comment%%`, `$math$`, and `$$block math$$` as literal text with their markers intact, and a callout (`> [!note]`) parses as an ordinary blockquote whose text is preserved.
Only the constructs consumers actually use get dedicated typed nodes: the three Annotations, Wikilink, Transclusion, and the core CommonMark blocks and inlines.
Anything remark *does* tokenise into a node core doesn't model (e.g. a GFM table, strikethrough) is preserved losslessly through a raw fallback — a **Raw Block** at block level, a **Raw Inline** at inline level.
Any construct can later be promoted (adding its plugin and/or a typed node) as an additive, non-breaking change once a consumer needs to recognise it.
See ADR 0005.
## Raw Block / Raw Inline
The fallback nodes that make [[Lossless OFM Parsing]] possible.
**Raw Block** carries the verbatim source of any block-level remark node not explicitly modelled by core's Block catalogue.
**Raw Inline** carries the verbatim source of any inline-level remark node not explicitly modelled by core's Inline Node catalogue.
They are safety valves, not targets: the implementation models every construct that appears in recipe fixtures explicitly and lets everything else fall through to raw.
The verbatim source is captured by **position-slicing** — slicing the original input string using the offsets remark records on each node — so the value is byte-for-byte what the author wrote and round-trips unchanged (never re-stringified/normalised).
A raw node exposes only that `value` string; it carries no `kind`/type hint (that would reintroduce a type vocabulary with no consumer yet). A construct is promoted to a dedicated typed node when a consumer needs its structure.
--- ---
## Open / Unresolved ## Open / Unresolved
- **Block node type catalogue** — the exact set of Block types core defines (heading, paragraph, list, code block, blockquote, etc.) and how unrecognised types are represented. - **Unit normalisation** for Ingredient/Cookware — the parser should normalise known aliases to a canonical abbreviation (`grams``g`, `kilograms``kg`); exact alias table and canonical forms are TBD. This is a parser concern, not a format constraint. (Timer unit normalisation is already fully specified above.)
- **Unit normalisation** — the parser should normalise known aliases to a canonical abbreviation (`grams``g`, `kilograms``kg`); exact alias table and canonical forms are TBD. This is a parser concern, not a format constraint.

View File

@@ -0,0 +1,57 @@
# ADR 0004 — Total-Function Parser with a Diagnostics Channel
**Status**: Accepted
## Context
`@kitchen-md/core`'s `parse` is a pure function from a Recipe File string to a Document AST.
Almost every input is forgiving by construction: CommonMark has no syntax errors, so any Markdown body is valid; unknown OFM constructs fall through to a raw fallback (see ADR 0005); and a malformed annotation such as bare `@foo` simply stays as plain text.
The one input class that can be genuinely invalid is **frontmatter YAML**.
`remark-frontmatter` only extracts the `---` block as a raw string; core still has to parse that string into the plain object the spec requires, and YAML can be syntactically broken (bad indentation, an unclosed quote, a tab where spaces are required).
Three approaches were considered for what `parse` does when the frontmatter YAML is invalid:
**Option A — Throw**
`parse` raises a typed error on invalid YAML.
The body never throws; only broken frontmatter does.
**Option B — Total function, silently empty**
Invalid YAML yields `frontmatter: {}` and parsing continues.
`parse` never throws.
**Option C — Total function with a diagnostics channel**
Invalid YAML yields `frontmatter: {}`, parsing continues, and a non-fatal warning is surfaced through a new top-level `diagnostics` array on the Document AST.
The raw invalid YAML and the parse error are preserved in the diagnostic.
`parse` never throws.
## Decision
**Option C**`parse` is a total function that never throws, and reports genuinely invalid input (currently only malformed frontmatter YAML) through a `diagnostics` array on the Document AST.
The Document AST top-level shape becomes `{ frontmatter, blocks, diagnostics }`.
`frontmatter` stays a plain `Record<string, unknown>` (empty when absent or invalid).
`diagnostics` is a list of `{ severity, code, message, source?, position? }`, empty in the normal case.
The invalid-frontmatter case emits `{ severity: "warning", code: "invalid-frontmatter", message, source: <raw YAML>, position }`.
## Rationale
Throwing (Option A) makes `parse` partial and pushes a `try`/`catch` onto every consumer for a condition that is not fatal to the rest of the document — the body is still perfectly parseable.
It also makes the CLI and the future Obsidian plugin each responsible for reconstructing a graceful-degradation story that the parser is better placed to provide once.
Silently emptying (Option B) is total but lossy: a consumer cannot distinguish "this file has no frontmatter" from "this file has broken frontmatter," so real authoring mistakes vanish without a trace.
Option C keeps the forgiving behaviour everywhere it makes sense while refusing to swallow the one class of input that is unambiguously the author's mistake.
It mirrors how Obsidian itself behaves: it still renders a note whose frontmatter is broken, and shows a warning rather than refusing the file.
A separate `diagnostics` array — rather than widening `frontmatter` into a discriminated union such as `{ valid: true, data } | { valid: false, raw, error }` — keeps the common path clean.
User story 2 wants frontmatter as "a plain object so I can read whatever fields I need"; a union would force every consumer to narrow on validity before reading a single field.
A diagnostics array leaves the happy path untouched and gives the parser a natural, extensible home for any future non-fatal warning.
## Consequences
- The Document AST gains a third top-level key, `diagnostics`, always present (empty when there is nothing to report).
- `parse` is total: no input causes it to throw. Consumers never need to wrap it in `try`/`catch`.
- Consumers decide how to surface diagnostics — the CLI can print a warning to stderr, the Obsidian plugin can show a banner — using the preserved `source` and `position`.
- `Diagnostic` is a public API type with a stable, machine-readable `code`. Adding new diagnostic codes later is additive and non-breaking.
- The only diagnostic emitted today is `invalid-frontmatter`; the channel exists to absorb future non-fatal parse issues without further shape changes.

View File

@@ -0,0 +1,54 @@
# ADR 0005 — Lossless OFM Parsing via Raw Fallbacks
**Status**: Accepted
## Context
The parser must accept **all** of Obsidian Flavored Markdown without ever failing or discarding source, so that the CLI and the future Obsidian plugin can rely on it for any real Recipe File.
OFM is large: on top of CommonMark it adds wikilinks, embeds, callouts, comments, footnotes, LaTeX math, diagrams, block references, tags, highlights, tables, task lists, and strikethrough.
ADR 0003 already established that core defines its own AST types and keeps a clean, minimal vocabulary scoped to what consumers actually need.
That leaves an open question: what happens to every OFM construct core does **not** model?
The spec's block catalogue included a `RawBlock` safety valve but no inline equivalent, so an unmodelled inline construct (a highlight, inline math, a footnote reference) had nowhere faithful to go and would silently collapse to its text, losing the markup.
Two approaches were considered:
**Option A — Model every OFM construct**
Give each construct its own first-class typed node: `CalloutBlock`, `MathNode`, `HighlightNode`, `TableBlock`, `TagNode`, `BlockRefNode`, and so on.
This requires a mature remark plugin for each and a large, growing public API.
**Option B — Lossless parse with raw fallbacks**
Model only the constructs consumers actually use (the three Annotations, Wikilink, Transclusion, and the core CommonMark blocks and inlines).
Preserve everything else losslessly through raw fallback nodes: a `RawBlock` at block level and a new `RawInline` at inline level, each carrying the original source verbatim.
## Decision
**Option B** — lossless parse with raw fallbacks.
- The public AST models the annotations, wikilinks, transclusions, and core CommonMark blocks/inlines explicitly.
- Any node remark produces that core does not model is preserved as `RawBlock` (block level) or `RawInline` (inline level).
- A raw node exposes a single `value` string and nothing else — no `kind` or type hint that would reintroduce a type vocabulary with no consumer.
- `value` is captured by **position-slicing**: the original input string is sliced using the offsets remark records on each node, so the value is byte-for-byte what the author wrote and round-trips unchanged. It is never re-stringified, which would normalise formatting and break losslessness.
- The required internal plugin set is deliberately minimal: `remark-parse` + `remark-frontmatter` + `remark-gfm` (tables, task lists, strikethrough, autolinks) + `remark-wiki-link` (links and embeds), plus core's annotation transform. Most other OFM syntax (`==highlight==`, `%%comment%%`, `$math$`, callouts) is already lossless as plain text or an ordinary blockquote, so it needs no dedicated plugin.
## Rationale
"The parser can parse all of OFM" is satisfied by lossless, never-fails handling; it does not require a typed node for every construct.
Option A would commit the project to modelling and maintaining a dozen node types — and their plugins, several of which are niche — with no consumer asking for any of them, in direct tension with ADR 0003's minimal-vocabulary goal.
Option B keeps the public API small today and stays fully faithful: a consumer that does not care about a construct ignores its raw node, and a consumer that wants to render it re-parses or re-renders the raw Markdown.
Any construct can be **promoted** from a raw fallback to a dedicated typed node (adding its plugin and structured fields) the day a consumer genuinely needs to distinguish it — an additive, non-breaking change.
Position-slicing rather than re-stringifying matters specifically for the round-trip case: the Obsidian plugin writing a document back out must not silently reformat every table, callout, and math block the parser did not model — including ones the user never touched.
Byte-for-byte preservation is the only guarantee that avoids that.
Leaving the raw node opaque (`value` only) avoids shipping a half-measure: a `kind` string is neither structured enough to be useful nor cheap enough to be free, since it forces core to define and maintain a construct-name vocabulary. Promotion to a real typed node is strictly more useful when the need arrives.
## Consequences
- Core defines a new `RawInline` node type alongside the existing `RawBlock`, closing the inline-fallback gap.
- Raw nodes are opaque verbatim source; consumers treat `value` as Markdown.
- The parser depends on `remark-gfm` and `remark-wiki-link` in addition to `remark-parse` and `remark-frontmatter`; adding plugins later to recognise more constructs is additive.
- Because raw values are position-sliced, the internal annotation transform must not invalidate the positions of the nodes that become raw. It only rewrites text nodes (which are always modelled, never raw), so raw nodes keep their original parse positions.
- Round-tripping a document preserves every unmodelled construct exactly as written.
- Raw fallbacks are a safety valve, not a target: the implementation models every construct that appears in recipe fixtures explicitly and lets only genuinely unmodelled syntax fall through.

10
SPEC.md
View File

@@ -57,8 +57,11 @@ Multi-word names work naturally: `@unsalted butter{30 g}`.
- Mixed number: `1 1/2` - Mixed number: `1 1/2`
**Unit**: optional, free-form string. **Unit**: optional, free-form string.
When both quantity and unit are present, they are delimited by the **last space** inside the braces. The quantity/unit split is **grammar-driven**: the parser matches the quantity greedily against the quantity grammar (mixed number → fraction → decimal → integer) anchored at the start of the braces, and any non-empty remainder after the delimiting space is the unit.
`@butter{1 1/2 tbsp}` → quantity `1 1/2`, unit `tbsp`. `@butter{1 1/2 tbsp}` → quantity `1 1/2`, unit `tbsp`.
`@butter{1 1/2}` → quantity `1 1/2`, no unit.
A plain "last space" rule cannot express this, because a mixed-number quantity itself contains a space; the grammar is authoritative.
If the brace content does not begin with a grammar-valid quantity (e.g. `@stock{a splash}`), the entire content is preserved as the quantity string and the unit is left empty, so no annotation is dropped.
**Examples**: **Examples**:
``` ```
@@ -87,7 +90,7 @@ $name{quantity unit}
**Quantity**: optional. Follows the same format rules as Ingredient quantity (integer, decimal, fraction, mixed number). **Quantity**: optional. Follows the same format rules as Ingredient quantity (integer, decimal, fraction, mixed number).
**Unit**: optional, free-form. Same last-space delimiter rule as Ingredient. **Unit**: optional, free-form. Same grammar-driven quantity/unit split as Ingredient.
**Examples**: **Examples**:
``` ```
@@ -220,4 +223,5 @@ step-num = 1*DIGIT
``` ```
> Note: `unit` in ingredient/cookware is free-form and not captured in the grammar above. > Note: `unit` in ingredient/cookware is free-form and not captured in the grammar above.
> The last-space rule applies at parse time: everything after the last space inside `{}` is the unit; everything before it is the quantity. > At parse time the quantity is matched greedily against the `quantity` production anchored at the start of `{}`; any non-empty remainder after the delimiting space is the unit.
> If the content does not begin with a grammar-valid `quantity`, the whole brace content becomes the quantity string and the unit is empty.