dev/docs/03 wrk/ariadne.md

# Ariadne design notes

## Purpose

These notes capture the design state from the Ariadne pipeline and Ren'Py AST discussion. They are intended to let a new session resume the design without relying on the discarded pull-request work.

Do not assume the implementation described here still exists in the Ariadne worktree. The pull request work was considered exploratory and should be dropped or redone cleanly.

## Core state model

Ariadne's core state should be graph-first. The current agreed shape is:

```rust
State {
    nodes: Map<NodeKey, Node>,
    edges: Map<NodeKey, Set<NodeKey>>,
    pending: Vec<Operation>,
}
```

Meaning:

- `nodes` is the committed node store.
- `edges` is committed directed connectivity.
- `pending` is a first-class operation log waiting to be applied.

The mutation vocabulary is expected to be an enum along these lines:

```rust
enum Operation {
    AddNode { key: NodeKey, node: Node },
    EditNode { key: NodeKey, patch: NodePatch },
    RemoveNode { key: NodeKey },
    AddEdge { from: NodeKey, to: NodeKey },
    RemoveEdge { from: NodeKey, to: NodeKey },
}
```

Stages should not directly mutate committed state. A stage should inspect the current state and emit operations. Applying pending operations advances the graph.

## Edges

Edges are simple directed connectivity for now:

```text
from -> to
```

A node may have multiple outgoing edges, so a literal `Map<NodeKey, NodeKey>` is not enough. Use a shape equivalent to:

```rust
Map<NodeKey, Set<NodeKey>>
```

Do not put conditional meaning on edges yet. The Python prototype solved conditional flow structurally:

- conditional expressions live on conditional nodes;
- menu choice conditions live on menu-choice or menu data;
- branch and merge edges are ordinary directed edges.

This means edge metadata is not currently required. If a future requirement needs multiple semantic edges between the same pair of nodes, revisit the edge model then.

## Node identity

Label names are not valid primary node keys. We checked combined source extracted from multiple launchers and found collisions, including `label start:` appearing in more than one source universe. There were dozens of duplicate label names across the combined prepared source.

The current direction is that `NodeKey` should include a namespace:

```rust
NodeKey {
    namespace,
    path,
    line,
    occurrence,
}
```

The namespace identifies the launcher/source universe. Edges may cross namespaces, because later launchers can depend on decisions or state from earlier launchers.

`path` and `line` identify the source location inside the namespace. `occurrence` became relevant after inspecting Ren'Py AST output: one physical source line can produce multiple AST nodes, for example display/transition nodes around the same line. If we use AST-derived nodes, line alone is not enough.

Earlier alternatives that were rejected or deferred:

- label-name keys, because labels collide across source universes;
- flat string keys without structure, because stages will need to reason about namespace and source identity;
- columns, because column-level identity is not currently useful;
- ranges, because the Python prototype did not model ranges, and multiline statements were rare in that prototype.

Multiline statements do exist. The Python prototype found a very small number and chose to treat them as physical-line parser limitations. For Ariadne, if AST ingestion is used, Ren'Py's own AST should handle logical statements better than our source parser.

## Dialogue blocks

A dialogue block is the Ren'Py/game unit counted by seen-dialogue coverage. For Ariadne, dialogue-block coverage maps mainly to unique Ren'Py `Say` statements.

Important identity rule:

```text
same dialogue text at different source locations = different dialogue blocks
```

So dialogue-block identity must include source identity. It must not be based only on speaker and text.

## Python prototype lessons

The previous Python prototype used `metadata.key` strings on parsed nodes. Its key assignment was approximately:

```python
if node is LabelNode:
    key = node.name
elif node is InitNode or InitPriorityOffsetNode:
    key = f"{path.stem}:init:{line_no}"
elif node has parent:
    key = f"{parent.key}:{line_no}"
else:
    key = str(line_no)
```

This was useful for a narrower source universe but is not enough for Ariadne's multi-launcher model.

The Python prototype also solved conditional and menu edges with structural graph logic:

- each label/init owns a subgraph;
- body statements are chained sequentially;
- conditionals use a stack of branch frames;
- branch tails merge into the closing successor;
- jump-terminated branches do not get false fallthrough merge edges;
- menus fan out to choices and fan back into a shared continuation.

Those ideas should inform future edge-building stages.

## Ren'Py AST ingestion decision

We investigated using Ren'Py AST as input instead of parsing decompiled `.rpy` source.

Direct Rust `.rpyc` pickle decoding partially worked:

```text
.rpyc -> RPYC slot table -> zlib -> pickle -> AST-like values
```

It succeeded on small/simple files, but failed on branch-heavy files with recursive pickle/object structure. The conclusion is not to make raw Ren'Py pickle decoding the immediate Rust path.

Preferred approach:

```text
launcher-bundled Python/Ren'Py runtime
  -> load .rpyc AST objects
  -> export normalized Ariadne-neutral JSON records
  -> Rust imports JSON into State operations
```

Ren'Py AST is an input format, not Ariadne's internal model.

## Ren'Py AST object shape

Ren'Py AST objects are Python objects from `renpy.ast`, with one class per statement-like construct. Common base fields include:

```python
node.filename
node.linenumber
node.name
node.next
```

Examples of useful node classes and fields:

```python
Label:
    name
    block
    parameters
    hide

Say:
    who
    what
    identifier
    attributes
    temporary_attributes
    arguments
    with_

If:
    entries  # list of (condition, block)

Menu:
    items    # list of (label_text, condition, block)

Jump:
    target
    expression

Call:
    label
    expression
    arguments

Python:
    code.source
    store
    hide
```

For dialogue coverage, `Say.identifier` and translation wrapper information may be important. Slot choice matters: slot 1 has the pre-static-transform AST; slot 2 can include translation wrappers such as `Translate` and `EndTranslate` around `Say` nodes.

## Prototype that was created and should be discarded

A prototype was created in the Ariadne worktree but should not be treated as the desired final shape:

```text
src/ariadne/prototypes/renpy-ast-adapter/
src/ariadne/tools/export-renpy-ast.py
```

It tested:

```text
.rpyc slot
  -> bundled Python adapter
  -> normalized AST JSON records
  -> Rust State { nodes, edges, pending }
```

The normalized JSON record shape was:

```json
{
  "key": {
    "namespace": "season-3",
    "path": "game/scripts/.../file.rpy",
    "line": 123,
    "occurrence": 0
  },
  "kind": "Say",
  "fields": {
    "who": "mc",
    "what": "...",
    "identifier": null
  }
}
```

The prototype successfully exported a branch-heavy `.rpyc` file that direct Rust pickle decoding could not handle. It showed that AST ingestion can produce thousands of nodes and hundreds of repeated source locations.

Why discard/rework it:

- it was added into the existing shell preparation script too quickly;
- the shell script became responsible for too much;
- Bazel runfiles handling made the shell script messy;
- the PR should be refocused before this work continues.

## Preparation output direction

Preparation output should evolve from:

```text
output/
  source/
  work/
  logs/
```

to:

```text
output/
  ast/      # normalized Ren'Py AST JSON
  source/   # decompiled/debug source
  work/     # extracted raw/intermediate files
  logs/
```

Decompiled source remains useful as a debug artifact, but may stop being the primary parser input.

## Preparation tooling direction

The shell script is becoming too broad. A better next step is to move preparation into its own Python tool before returning to Ariadne core.

That preparation tool should own:

- game-root and launcher discovery;
- archive extraction;
- `.rpyc` collection;
- AST export through the matching bundled Ren'Py runtime;
- optional decompilation for debug source;
- output layout and logs.

Ariadne core should only depend on the preparation tool's normalized artifacts.

## Python dependencies and vendoring

For this repository, "vendor" means making a dependency available behind a public `//third_party/...` target, like Rust crates are exposed through `//third_party/crates`.

If the preparation tool depends on `rpycdec`, Ariadne code should depend on a public alias such as:

```text
//third_party/python:rpycdec
```

not directly on the underlying source, pip repository, or external workspace.

The exact Bazel target shape for `rpycdec` is undecided because the actual `rpycdec` source has not been inspected in this session. The grounded next step is to inspect it first, then decide whether it should be wrapped as a `py_library`, `py_binary`, external archive, or another `//third_party` target.

The AST exporter itself needs only the Python standard library plus the target launcher's bundled Ren'Py runtime. The broader preparation flow still needs `rpycdec` for archive extraction and decompiled debug source unless that responsibility changes.

## Recommended next session start

1. Do not continue from the discarded PR implementation.
2. Inspect the actual `rpycdec` source and decide how to expose it under `//third_party/python` or another `//third_party` package.
3. Design a standalone preparation Python tool with no unnecessary external dependencies.
4. Keep the normalized AST JSON boundary between bundled Ren'Py runtime code and Rust Ariadne core.
5. Return to the Rust `State` shape after the preparation artifact contract is clearer.