alexion
3ffd2b3f9f
Capture the design decisions reached while grilling the core-parser spec: - Parser is a total function; invalid frontmatter YAML surfaces through a new Document AST `diagnostics` channel instead of throwing (ADR 0004). - Parse all of OFM losslessly via `RawBlock`/`RawInline` fallbacks with position-sliced verbatim source and a minimal plugin set (ADR 0005). - Grammar-driven (not last-space) quantity/unit split, fixing the mixed-number collision, with trimming and multi-word unit rules. - Container blocks nest `Block[]` so annotations surface inside blockquotes and list items; unified wikilink/transclusion shape. Update SPEC.md and CONTEXT.md accordingly; no parser code yet.
55 lines
5.1 KiB
Markdown
55 lines
5.1 KiB
Markdown
# ADR 0005 — Lossless OFM Parsing via Raw Fallbacks
|
|
|
|
**Status**: Accepted
|
|
|
|
## Context
|
|
|
|
The parser must accept **all** of Obsidian Flavored Markdown without ever failing or discarding source, so that the CLI and the future Obsidian plugin can rely on it for any real Recipe File.
|
|
OFM is large: on top of CommonMark it adds wikilinks, embeds, callouts, comments, footnotes, LaTeX math, diagrams, block references, tags, highlights, tables, task lists, and strikethrough.
|
|
|
|
ADR 0003 already established that core defines its own AST types and keeps a clean, minimal vocabulary scoped to what consumers actually need.
|
|
That leaves an open question: what happens to every OFM construct core does **not** model?
|
|
The spec's block catalogue included a `RawBlock` safety valve but no inline equivalent, so an unmodelled inline construct (a highlight, inline math, a footnote reference) had nowhere faithful to go and would silently collapse to its text, losing the markup.
|
|
|
|
Two approaches were considered:
|
|
|
|
**Option A — Model every OFM construct**
|
|
Give each construct its own first-class typed node: `CalloutBlock`, `MathNode`, `HighlightNode`, `TableBlock`, `TagNode`, `BlockRefNode`, and so on.
|
|
This requires a mature remark plugin for each and a large, growing public API.
|
|
|
|
**Option B — Lossless parse with raw fallbacks**
|
|
Model only the constructs consumers actually use (the three Annotations, Wikilink, Transclusion, and the core CommonMark blocks and inlines).
|
|
Preserve everything else losslessly through raw fallback nodes: a `RawBlock` at block level and a new `RawInline` at inline level, each carrying the original source verbatim.
|
|
|
|
## Decision
|
|
|
|
**Option B** — lossless parse with raw fallbacks.
|
|
|
|
- The public AST models the annotations, wikilinks, transclusions, and core CommonMark blocks/inlines explicitly.
|
|
- Any node remark produces that core does not model is preserved as `RawBlock` (block level) or `RawInline` (inline level).
|
|
- A raw node exposes a single `value` string and nothing else — no `kind` or type hint that would reintroduce a type vocabulary with no consumer.
|
|
- `value` is captured by **position-slicing**: the original input string is sliced using the offsets remark records on each node, so the value is byte-for-byte what the author wrote and round-trips unchanged. It is never re-stringified, which would normalise formatting and break losslessness.
|
|
- The required internal plugin set is deliberately minimal: `remark-parse` + `remark-frontmatter` + `remark-gfm` (tables, task lists, strikethrough, autolinks) + `remark-wiki-link` (links and embeds), plus core's annotation transform. Most other OFM syntax (`==highlight==`, `%%comment%%`, `$math$`, callouts) is already lossless as plain text or an ordinary blockquote, so it needs no dedicated plugin.
|
|
|
|
## Rationale
|
|
|
|
"The parser can parse all of OFM" is satisfied by lossless, never-fails handling; it does not require a typed node for every construct.
|
|
Option A would commit the project to modelling and maintaining a dozen node types — and their plugins, several of which are niche — with no consumer asking for any of them, in direct tension with ADR 0003's minimal-vocabulary goal.
|
|
|
|
Option B keeps the public API small today and stays fully faithful: a consumer that does not care about a construct ignores its raw node, and a consumer that wants to render it re-parses or re-renders the raw Markdown.
|
|
Any construct can be **promoted** from a raw fallback to a dedicated typed node (adding its plugin and structured fields) the day a consumer genuinely needs to distinguish it — an additive, non-breaking change.
|
|
|
|
Position-slicing rather than re-stringifying matters specifically for the round-trip case: the Obsidian plugin writing a document back out must not silently reformat every table, callout, and math block the parser did not model — including ones the user never touched.
|
|
Byte-for-byte preservation is the only guarantee that avoids that.
|
|
|
|
Leaving the raw node opaque (`value` only) avoids shipping a half-measure: a `kind` string is neither structured enough to be useful nor cheap enough to be free, since it forces core to define and maintain a construct-name vocabulary. Promotion to a real typed node is strictly more useful when the need arrives.
|
|
|
|
## Consequences
|
|
|
|
- Core defines a new `RawInline` node type alongside the existing `RawBlock`, closing the inline-fallback gap.
|
|
- Raw nodes are opaque verbatim source; consumers treat `value` as Markdown.
|
|
- The parser depends on `remark-gfm` and `remark-wiki-link` in addition to `remark-parse` and `remark-frontmatter`; adding plugins later to recognise more constructs is additive.
|
|
- Because raw values are position-sliced, the internal annotation transform must not invalidate the positions of the nodes that become raw. It only rewrites text nodes (which are always modelled, never raw), so raw nodes keep their original parse positions.
|
|
- Round-tripping a document preserves every unmodelled construct exactly as written.
|
|
- Raw fallbacks are a safety valve, not a target: the implementation models every construct that appears in recipe fixtures explicitly and lets only genuinely unmodelled syntax fall through.
|