dev/docs/03 wrk/ariadne.md

Ariadne design notes

Purpose

These notes capture the design state from the Ariadne pipeline and Ren'Py AST discussion. They are intended to let a new session resume the design without relying on the discarded pull-request work.

Do not assume the implementation described here still exists in the Ariadne worktree. The pull request work was considered exploratory and should be dropped or redone cleanly.

Core state model

Ariadne's core state should be graph-first. The current agreed shape is:

State {
    nodes: Map<NodeKey, Node>,
    edges: Map<NodeKey, Set<NodeKey>>,
    pending: Vec<Operation>,
}

Meaning:

• nodes is the committed node store.
• edges is committed directed connectivity.
• pending is a first-class operation log waiting to be applied.

The mutation vocabulary is expected to be an enum along these lines:

enum Operation {
    AddNode { key: NodeKey, node: Node },
    EditNode { key: NodeKey, patch: NodePatch },
    RemoveNode { key: NodeKey },
    AddEdge { from: NodeKey, to: NodeKey },
    RemoveEdge { from: NodeKey, to: NodeKey },
}

Stages should not directly mutate committed state. A stage should inspect the current state and emit operations. Applying pending operations advances the graph.

Edges

Edges are simple directed connectivity for now:

from -> to

A node may have multiple outgoing edges, so a literal Map<NodeKey, NodeKey> is not enough. Use a shape equivalent to:

Map<NodeKey, Set<NodeKey>>

Do not put conditional meaning on edges yet. The Python prototype solved conditional flow structurally:

• conditional expressions live on conditional nodes;
• menu choice conditions live on menu-choice or menu data;
• branch and merge edges are ordinary directed edges.

This means edge metadata is not currently required. If a future requirement needs multiple semantic edges between the same pair of nodes, revisit the edge model then.

Node identity

Label names are not valid primary node keys. We checked combined source extracted from multiple launchers and found collisions, including label start: appearing in more than one source universe. There were dozens of duplicate label names across the combined prepared source.

The current direction is that NodeKey should include a namespace:

NodeKey {
    namespace,
    path,
    line,
    occurrence,
}

The namespace identifies the launcher/source universe. Edges may cross namespaces, because later launchers can depend on decisions or state from earlier launchers.

path and line identify the source location inside the namespace. occurrence became relevant after inspecting Ren'Py AST output: one physical source line can produce multiple AST nodes, for example display/transition nodes around the same line. If we use AST-derived nodes, line alone is not enough.

Earlier alternatives that were rejected or deferred:

• label-name keys, because labels collide across source universes;
• flat string keys without structure, because stages will need to reason about namespace and source identity;
• columns, because column-level identity is not currently useful;
• ranges, because the Python prototype did not model ranges, and multiline statements were rare in that prototype.

Multiline statements do exist. The Python prototype found a very small number and chose to treat them as physical-line parser limitations. For Ariadne, if AST ingestion is used, Ren'Py's own AST should handle logical statements better than our source parser.

Dialogue blocks

A dialogue block is the Ren'Py/game unit counted by seen-dialogue coverage. For Ariadne, dialogue-block coverage maps mainly to unique Ren'Py Say statements.

Important identity rule:

same dialogue text at different source locations = different dialogue blocks

So dialogue-block identity must include source identity. It must not be based only on speaker and text.

Python prototype lessons

The previous Python prototype used metadata.key strings on parsed nodes. Its key assignment was approximately:

if node is LabelNode:
    key = node.name
elif node is InitNode or InitPriorityOffsetNode:
    key = f"{path.stem}:init:{line_no}"
elif node has parent:
    key = f"{parent.key}:{line_no}"
else:
    key = str(line_no)

This was useful for a narrower source universe but is not enough for Ariadne's multi-launcher model.

The Python prototype also solved conditional and menu edges with structural graph logic:

• each label/init owns a subgraph;
• body statements are chained sequentially;
• conditionals use a stack of branch frames;
• branch tails merge into the closing successor;
• jump-terminated branches do not get false fallthrough merge edges;
• menus fan out to choices and fan back into a shared continuation.

Those ideas should inform future edge-building stages.

Ren'Py AST ingestion decision

We investigated using Ren'Py AST as input instead of parsing decompiled .rpy source.

Direct Rust .rpyc pickle decoding partially worked:

.rpyc -> RPYC slot table -> zlib -> pickle -> AST-like values

It succeeded on small/simple files, but failed on branch-heavy files with recursive pickle/object structure. The conclusion is not to make raw Ren'Py pickle decoding the immediate Rust path.

Preferred approach:

launcher-bundled Python/Ren'Py runtime
  -> load .rpyc AST objects
  -> export normalized Ariadne-neutral JSON records
  -> Rust imports JSON into State operations

Ren'Py AST is an input format, not Ariadne's internal model.

Ren'Py AST object shape

Ren'Py AST objects are Python objects from renpy.ast, with one class per statement-like construct. Common base fields include:

node.filename
node.linenumber
node.name
node.next

Examples of useful node classes and fields:

Label:
    name
    block
    parameters
    hide

Say:
    who
    what
    identifier
    attributes
    temporary_attributes
    arguments
    with_

If:
    entries  # list of (condition, block)

Menu:
    items    # list of (label_text, condition, block)

Jump:
    target
    expression

Call:
    label
    expression
    arguments

Python:
    code.source
    store
    hide

For dialogue coverage, Say.identifier and translation wrapper information may be important. Slot choice matters: slot 1 has the pre-static-transform AST; slot 2 can include translation wrappers such as Translate and EndTranslate around Say nodes.

Prototype that was created and should be discarded

A prototype was created in the Ariadne worktree but should not be treated as the desired final shape:

src/ariadne/prototypes/renpy-ast-adapter/
src/ariadne/tools/export-renpy-ast.py

It tested:

.rpyc slot
  -> bundled Python adapter
  -> normalized AST JSON records
  -> Rust State { nodes, edges, pending }

The normalized JSON record shape was:

{
  "key": {
    "namespace": "season-3",
    "path": "game/scripts/.../file.rpy",
    "line": 123,
    "occurrence": 0
  },
  "kind": "Say",
  "fields": {
    "who": "mc",
    "what": "...",
    "identifier": null
  }
}

The prototype successfully exported a branch-heavy .rpyc file that direct Rust pickle decoding could not handle. It showed that AST ingestion can produce thousands of nodes and hundreds of repeated source locations.

Why discard/rework it:

• it was added into the existing shell preparation script too quickly;
• the shell script became responsible for too much;
• Bazel runfiles handling made the shell script messy;
• the PR should be refocused before this work continues.

Clew artifact boundary

Clew now owns extraction, preparation, and Ren'Py AST export. Ariadne owns analysis over Clew-produced data.

Ariadne consumes Clew's raw clew-renpy-ast-v1 JSON directly. It deserializes Clew's generic serializer format into typed Rust structs and enums. It then maps the full Clew AST into Ariadne-owned internal data structures.

The first importer preserves unrecognized or unneeded Ren'Py objects. Use an UnknownNode or equivalent representation instead of dropping data. Semantic helpers can exist for known Ren'Py statement classes.

NodeKey.namespace comes from the Clew AST file path relative to output/ast/. Clew preserves the game-root boundary in that path. This keeps label names from different source universes separate.

The first Ariadne CLI prints a human-readable validation report by default. The report includes counts for documents, namespaces, and nodes by kind. It also includes counts for unknown nodes, edges, refs, and warnings. The CLI also supports an opt-in debug dump of internal state, for example --dump-state. Treat that dump as a development surface until Ariadne documents a stable format.

Route planning is out of scope for the first importer slice.

Preparation output direction

Clew produces the preparation output for Ariadne. The current relevant output is:

output/
  ast/        # Clew `clew-renpy-ast-v1` JSON
  extracted/  # extracted game files

Decompiled source is no longer Ariadne's primary input. The old src/ariadne/extract-and-decompile.sh script was obsolete. Ariadne now relies on Clew for extraction and preparation.

Python dependencies and vendoring

For this repository, "vendor" means making a dependency available behind a public //third_party/... target, like Rust crates are exposed through //third_party/crates.

If the preparation tool depends on rpycdec, Ariadne code should depend on a public alias such as:

//third_party/python:rpycdec

not directly on the underlying source, pip repository, or external workspace.

The exact Bazel target shape for rpycdec is undecided because the actual rpycdec source has not been inspected in this session. The grounded next step is to inspect it first, then decide whether it should be wrapped as a py_library, py_binary, external archive, or another //third_party target.

The AST exporter itself needs only the Python standard library plus the target launcher's bundled Ren'Py runtime. The broader preparation flow still needs rpycdec for archive extraction and decompiled debug source unless that responsibility changes.

Recommended next session start

1. Do not continue from the discarded PR implementation.
2. Implement typed Rust deserialization for Clew's clew-renpy-ast-v1 JSON.
3. Map every Clew AST value into Ariadne-owned internal data structures.
4. Preserve unknown Ren'Py objects as UnknownNode or an equivalent node.
5. Build the default validation report for imported Clew output.
6. Add an opt-in debug dump for internal state.