dev/docs/03 wrk/ariadne.md

# Ariadne design notes

## Purpose

These notes capture the design state from the Ariadne pipeline and Ren'Py AST discussion. They are intended to let a new session resume the design without relying on the discarded pull-request work.

Do not assume the implementation described here still exists in the Ariadne worktree. The pull request work was considered exploratory and should be dropped or redone cleanly.

## Product goals

Ariadne analyzes a local Ren'Py game install.
It builds a source graph and plans routes through target nodes.

Goal order:

1. Unlock all Steam achievements.
2. Maximize Season 3 dialogue coverage.
3. Extend dialogue coverage to Seasons 1 and 2.

Achievement unlocks appear in source as reachable calls.
The first planning target is each achievement grant node.

Season 3 exposes an in-game percentage for seen dialogue.
That percentage gives a validation signal for Ariadne's coverage model.

The first phase ignores player state.
It computes routes from source alone.
A later phase reads save files or persistent data.
It removes completed targets and plans remaining coverage.

## Privacy constraint

Do not write the target game's title in committed material.
Use `target game` or `Ren'Py game` in notes, code, commits, and pull requests.
Keep title-specific paths and names in ignored local data only.

## Core state model

Ariadne's core state should be graph-first. The current agreed shape is:

```rust
State {
    nodes: Map<NodeKey, Node>,
    edges: Map<NodeKey, Set<NodeKey>>,
    pending: Vec<Operation>,
}
```

Meaning:

- `nodes` is the committed node store.
- `edges` is committed directed connectivity.
- `pending` is a first-class operation log waiting to be applied.

The mutation vocabulary is expected to be an enum along these lines:

```rust
enum Operation {
    AddNode { key: NodeKey, node: Node },
    EditNode { key: NodeKey, patch: NodePatch },
    RemoveNode { key: NodeKey },
    AddEdge { from: NodeKey, to: NodeKey },
    RemoveEdge { from: NodeKey, to: NodeKey },
}
```

Stages should not directly mutate committed state. A stage should inspect the current state and emit operations. Applying pending operations advances the graph.

## Edges

Edges are simple directed connectivity for now:

```text
from -> to
```

A node may have multiple outgoing edges, so a literal `Map<NodeKey, NodeKey>` is not enough. Use a shape equivalent to:

```rust
Map<NodeKey, Set<NodeKey>>
```

Do not put conditional meaning on edges yet. The Python prototype solved conditional flow structurally:

- conditional expressions live on conditional nodes;
- menu choice conditions live on menu-choice or menu data;
- branch and merge edges are ordinary directed edges.

This means edge metadata is not currently required. If a future requirement needs multiple semantic edges between the same pair of nodes, revisit the edge model then.

## Node identity

Label names are not valid primary node keys. We checked combined source extracted from multiple launchers and found collisions, including `label start:` appearing in more than one source universe. There were dozens of duplicate label names across the combined prepared source.

The current direction is that `NodeKey` should include a namespace:

```rust
NodeKey {
    namespace,
    path,
    line,
    occurrence,
}
```

The namespace identifies the launcher/source universe. Edges may cross namespaces, because later launchers can depend on decisions or state from earlier launchers.

`path` and `line` identify the source location inside the namespace. `occurrence` became relevant after inspecting Ren'Py AST output: one physical source line can produce multiple AST nodes, for example display/transition nodes around the same line. If we use AST-derived nodes, line alone is not enough.

Earlier alternatives that were rejected or deferred:

- label-name keys, because labels collide across source universes;
- flat string keys without structure, because stages will need to reason about namespace and source identity;
- columns, because column-level identity is not currently useful;
- ranges, because the Python prototype did not model ranges, and multiline statements were rare in that prototype.

Multiline statements do exist. The Python prototype found a very small number and chose to treat them as physical-line parser limitations. For Ariadne, if AST ingestion is used, Ren'Py's own AST should handle logical statements better than our source parser.

## Dialogue blocks

A dialogue block is the Ren'Py/game unit counted by seen-dialogue coverage. For Ariadne, dialogue-block coverage maps mainly to unique Ren'Py `Say` statements.

Important identity rule:

```text
same dialogue text at different source locations = different dialogue blocks
```

So dialogue-block identity must include source identity. It must not be based only on speaker and text.

## Python prototype lessons

The previous Python prototype used `metadata.key` strings on parsed nodes. Its key assignment was approximately:

```python
if node is LabelNode:
    key = node.name
elif node is InitNode or InitPriorityOffsetNode:
    key = f"{path.stem}:init:{line_no}"
elif node has parent:
    key = f"{parent.key}:{line_no}"
else:
    key = str(line_no)
```

This was useful for a narrower source universe but is not enough for Ariadne's multi-launcher model.

The Python prototype also solved conditional and menu edges with structural graph logic:

- each label/init owns a subgraph;
- body statements are chained sequentially;
- conditionals use a stack of branch frames;
- branch tails merge into the closing successor;
- jump-terminated branches do not get false fallthrough merge edges;
- menus fan out to choices and fan back into a shared continuation.

Those ideas should inform future edge-building stages.

## Ren'Py AST ingestion decision

We investigated using Ren'Py AST as input instead of parsing decompiled `.rpy` source.

Direct Rust `.rpyc` pickle decoding partially worked:

```text
.rpyc -> RPYC slot table -> zlib -> pickle -> AST-like values
```

It succeeded on small/simple files, but failed on branch-heavy files with recursive pickle/object structure. The conclusion is not to make raw Ren'Py pickle decoding the immediate Rust path.

Preferred approach:

```text
launcher-bundled Python/Ren'Py runtime
  -> load .rpyc AST objects
  -> export normalized Ariadne-neutral JSON records
  -> Rust imports JSON into State operations
```

Ren'Py AST is an input format, not Ariadne's internal model.

## Ren'Py AST object shape

Ren'Py AST objects are Python objects from `renpy.ast`, with one class per statement-like construct. Common base fields include:

```python
node.filename
node.linenumber
node.name
node.next
```

Examples of useful node classes and fields:

```python
Label:
    name
    block
    parameters
    hide

Say:
    who
    what
    identifier
    attributes
    temporary_attributes
    arguments
    with_

If:
    entries  # list of (condition, block)

Menu:
    items    # list of (label_text, condition, block)

Jump:
    target
    expression

Call:
    label
    expression
    arguments

Python:
    code.source
    store
    hide
```

For dialogue coverage, `Say.identifier` and translation wrapper information may be important. Slot choice matters: slot 1 has the pre-static-transform AST; slot 2 can include translation wrappers such as `Translate` and `EndTranslate` around `Say` nodes.

## Prototype that was created and should be discarded

A prototype was created in the Ariadne worktree but should not be treated as the desired final shape:

```text
src/ariadne/prototypes/renpy-ast-adapter/
src/ariadne/tools/export-renpy-ast.py
```

It tested:

```text
.rpyc slot
  -> bundled Python adapter
  -> normalized AST JSON records
  -> Rust State { nodes, edges, pending }
```

The normalized JSON record shape was:

```json
{
  "key": {
    "namespace": "season-3",
    "path": "game/scripts/.../file.rpy",
    "line": 123,
    "occurrence": 0
  },
  "kind": "Say",
  "fields": {
    "who": "mc",
    "what": "...",
    "identifier": null
  }
}
```

The prototype successfully exported a branch-heavy `.rpyc` file that direct Rust pickle decoding could not handle. It showed that AST ingestion can produce thousands of nodes and hundreds of repeated source locations.

Why discard/rework it:

- it was added into the existing shell preparation script too quickly;
- the shell script became responsible for too much;
- Bazel runfiles handling made the shell script messy;
- the PR should be refocused before this work continues.

## Clew artifact boundary

Clew now owns extraction, preparation, and Ren'Py AST export.
Ariadne owns analysis over Clew-produced data.

Ariadne consumes Clew's raw `clew-renpy-ast-v1` JSON directly.
It deserializes Clew's generic serializer format into typed Rust structs and enums.
It then maps the full Clew AST into Ariadne-owned internal data structures.

The first importer preserves unrecognized or unneeded Ren'Py objects.
Use an `UnknownNode` or equivalent representation instead of dropping data.
Semantic helpers can exist for known Ren'Py statement classes.

`NodeKey.namespace` comes from the Clew AST file path relative to `output/ast/`.
Clew preserves the game-root boundary in that path.
This keeps label names from different source universes separate.

The first Ariadne CLI prints a human-readable validation report by default.
The report includes counts for documents, namespaces, and nodes by kind.
It also includes counts for unknown nodes, edges, refs, and warnings.
The CLI also supports an opt-in debug dump of internal state, for example `--dump-state`.
Treat that dump as a development surface until Ariadne documents a stable format.

Route planning is out of scope for the first importer slice.

## Pipeline stage boundary

A library stage is a pure state transformation.
It receives committed `State` and stage-specific inputs.
It returns a new `State` with pending operations.

A CLI stage is a user-facing execution target.
It owns filesystem discovery, file reads, and stage orchestration.
It can deserialize Clew JSON by calling Ariadne library helpers.

`run_parse` receives already-deserialized Clew documents.
It does not discover files, read files, or deserialize JSON.

The runner applies pending operations after each library stage.
The next stage receives committed state with no pending operations.

`apply_pending` returns a new `State`.
It applies the pending batch atomically.
Invalid pending operations do not produce partial committed state.

The first implementation does not resume from serialized stage artifacts.
It also avoids a stage trait, shared context object, and stage invocation enum.
Give each stage function only the inputs it needs.

## Preparation output direction

Clew produces the preparation output for Ariadne.
The current relevant output is:

```text
output/
  ast/        # Clew `clew-renpy-ast-v1` JSON
  extracted/  # extracted game files
```

Decompiled source is no longer Ariadne's primary input.
The old `src/ariadne/extract-and-decompile.sh` script was obsolete.
Ariadne now relies on Clew for extraction and preparation.

## Python dependencies and vendoring

For this repository, "vendor" means making a dependency available behind a public `//third_party/...` target, like Rust crates are exposed through `//third_party/crates`.

If the preparation tool depends on `rpycdec`, Ariadne code should depend on a public alias such as:

```text
//third_party/python:rpycdec
```

not directly on the underlying source, pip repository, or external workspace.

The exact Bazel target shape for `rpycdec` is undecided because the actual `rpycdec` source has not been inspected in this session. The grounded next step is to inspect it first, then decide whether it should be wrapped as a `py_library`, `py_binary`, external archive, or another `//third_party` target.

The AST exporter itself needs only the Python standard library plus the target launcher's bundled Ren'Py runtime. The broader preparation flow still needs `rpycdec` for archive extraction and decompiled debug source unless that responsibility changes.

## Recommended next session start

1. Do not continue from the discarded PR implementation.
2. Implement typed Rust deserialization for Clew's `clew-renpy-ast-v1` JSON.
3. Map every Clew AST value into Ariadne-owned internal data structures.
4. Preserve unknown Ren'Py objects as `UnknownNode` or an equivalent node.
5. Build the default validation report for imported Clew output.
6. Add an opt-in debug dump for internal state.