alexion/dev
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add ariadne planning notes

Browse Source
This commit was merged in pull request #26.
This commit is contained in:
alexion
2026-09-08 15:38:36 -04:00
parent c03becfbb9
commit af244d1064
4 changed files with 388 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

312
docs/03 wrk/ariadne.md Normal file
View File

@@ -0,0 +1,312 @@
# Ariadne design notes
## Purpose
These notes capture the design state from the Ariadne pipeline and Ren'Py AST discussion. They are intended to let a new session resume the design without relying on the discarded pull-request work.
Do not assume the implementation described here still exists in the Ariadne worktree. The pull request work was considered exploratory and should be dropped or redone cleanly.
## Core state model
Ariadne's core state should be graph-first. The current agreed shape is:
```rust
State {
nodes: Map<NodeKey, Node>,
edges: Map<NodeKey, Set<NodeKey>>,
pending: Vec<Operation>,
}
```
Meaning:
- `nodes` is the committed node store.
- `edges` is committed directed connectivity.
- `pending` is a first-class operation log waiting to be applied.
The mutation vocabulary is expected to be an enum along these lines:
```rust
enum Operation {
AddNode { key: NodeKey, node: Node },
EditNode { key: NodeKey, patch: NodePatch },
RemoveNode { key: NodeKey },
AddEdge { from: NodeKey, to: NodeKey },
RemoveEdge { from: NodeKey, to: NodeKey },
}
```
Stages should not directly mutate committed state. A stage should inspect the current state and emit operations. Applying pending operations advances the graph.
## Edges
Edges are simple directed connectivity for now:
```text
from -> to
```
A node may have multiple outgoing edges, so a literal `Map<NodeKey, NodeKey>` is not enough. Use a shape equivalent to:
```rust
Map<NodeKey, Set<NodeKey>>
```
Do not put conditional meaning on edges yet. The Python prototype solved conditional flow structurally:
- conditional expressions live on conditional nodes;
- menu choice conditions live on menu-choice or menu data;
- branch and merge edges are ordinary directed edges.
This means edge metadata is not currently required. If a future requirement needs multiple semantic edges between the same pair of nodes, revisit the edge model then.
## Node identity
Label names are not valid primary node keys. We checked combined source extracted from multiple launchers and found collisions, including `label start:` appearing in more than one source universe. There were dozens of duplicate label names across the combined prepared source.
The current direction is that `NodeKey` should include a namespace:
```rust
NodeKey {
namespace,
path,
line,
occurrence,
}
```
The namespace identifies the launcher/source universe. Edges may cross namespaces, because later launchers can depend on decisions or state from earlier launchers.
`path` and `line` identify the source location inside the namespace. `occurrence` became relevant after inspecting Ren'Py AST output: one physical source line can produce multiple AST nodes, for example display/transition nodes around the same line. If we use AST-derived nodes, line alone is not enough.
Earlier alternatives that were rejected or deferred:
- label-name keys, because labels collide across source universes;
- flat string keys without structure, because stages will need to reason about namespace and source identity;
- columns, because column-level identity is not currently useful;
- ranges, because the Python prototype did not model ranges, and multiline statements were rare in that prototype.
Multiline statements do exist. The Python prototype found a very small number and chose to treat them as physical-line parser limitations. For Ariadne, if AST ingestion is used, Ren'Py's own AST should handle logical statements better than our source parser.
## Dialogue blocks
A dialogue block is the Ren'Py/game unit counted by seen-dialogue coverage. For Ariadne, dialogue-block coverage maps mainly to unique Ren'Py `Say` statements.
Important identity rule:
```text
same dialogue text at different source locations = different dialogue blocks
```
So dialogue-block identity must include source identity. It must not be based only on speaker and text.
## Python prototype lessons
The previous Python prototype used `metadata.key` strings on parsed nodes. Its key assignment was approximately:
```python
if node is LabelNode:
key = node.name
elif node is InitNode or InitPriorityOffsetNode:
key = f"{path.stem}:init:{line_no}"
elif node has parent:
key = f"{parent.key}:{line_no}"
else:
key = str(line_no)
```
This was useful for a narrower source universe but is not enough for Ariadne's multi-launcher model.
The Python prototype also solved conditional and menu edges with structural graph logic:
- each label/init owns a subgraph;
- body statements are chained sequentially;
- conditionals use a stack of branch frames;
- branch tails merge into the closing successor;
- jump-terminated branches do not get false fallthrough merge edges;
- menus fan out to choices and fan back into a shared continuation.
Those ideas should inform future edge-building stages.
## Ren'Py AST ingestion decision
We investigated using Ren'Py AST as input instead of parsing decompiled `.rpy` source.
Direct Rust `.rpyc` pickle decoding partially worked:
```text
.rpyc -> RPYC slot table -> zlib -> pickle -> AST-like values
```
It succeeded on small/simple files, but failed on branch-heavy files with recursive pickle/object structure. The conclusion is not to make raw Ren'Py pickle decoding the immediate Rust path.
Preferred approach:
```text
launcher-bundled Python/Ren'Py runtime
-> load .rpyc AST objects
-> export normalized Ariadne-neutral JSON records
-> Rust imports JSON into State operations
```
Ren'Py AST is an input format, not Ariadne's internal model.
## Ren'Py AST object shape
Ren'Py AST objects are Python objects from `renpy.ast`, with one class per statement-like construct. Common base fields include:
```python
node.filename
node.linenumber
node.name
node.next
```
Examples of useful node classes and fields:
```python
Label:
name
block
parameters
hide
Say:
who
what
identifier
attributes
temporary_attributes
arguments
with_
If:
entries # list of (condition, block)
Menu:
items # list of (label_text, condition, block)
Jump:
target
expression
Call:
label
expression
arguments
Python:
code.source
store
hide
```
For dialogue coverage, `Say.identifier` and translation wrapper information may be important. Slot choice matters: slot 1 has the pre-static-transform AST; slot 2 can include translation wrappers such as `Translate` and `EndTranslate` around `Say` nodes.
## Prototype that was created and should be discarded
A prototype was created in the Ariadne worktree but should not be treated as the desired final shape:
```text
src/ariadne/prototypes/renpy-ast-adapter/
src/ariadne/tools/export-renpy-ast.py
```
It tested:
```text
.rpyc slot
-> bundled Python adapter
-> normalized AST JSON records
-> Rust State { nodes, edges, pending }
```
The normalized JSON record shape was:
```json
{
"key": {
"namespace": "season-3",
"path": "game/scripts/.../file.rpy",
"line": 123,
"occurrence": 0
},
"kind": "Say",
"fields": {
"who": "mc",
"what": "...",
"identifier": null
}
}
```
The prototype successfully exported a branch-heavy `.rpyc` file that direct Rust pickle decoding could not handle. It showed that AST ingestion can produce thousands of nodes and hundreds of repeated source locations.
Why discard/rework it:
- it was added into the existing shell preparation script too quickly;
- the shell script became responsible for too much;
- Bazel runfiles handling made the shell script messy;
- the PR should be refocused before this work continues.
## Preparation output direction
Preparation output should evolve from:
```text
output/
source/
work/
logs/
```
to:
```text
output/
ast/ # normalized Ren'Py AST JSON
source/ # decompiled/debug source
work/ # extracted raw/intermediate files
logs/
```
Decompiled source remains useful as a debug artifact, but may stop being the primary parser input.
## Preparation tooling direction
The shell script is becoming too broad. A better next step is to move preparation into its own Python tool before returning to Ariadne core.
That preparation tool should own:
- game-root and launcher discovery;
- archive extraction;
- `.rpyc` collection;
- AST export through the matching bundled Ren'Py runtime;
- optional decompilation for debug source;
- output layout and logs.
Ariadne core should only depend on the preparation tool's normalized artifacts.
## Python dependencies and vendoring
For this repository, "vendor" means making a dependency available behind a public `//third_party/...` target, like Rust crates are exposed through `//third_party/crates`.
If the preparation tool depends on `rpycdec`, Ariadne code should depend on a public alias such as:
```text
//third_party/python:rpycdec
```
not directly on the underlying source, pip repository, or external workspace.
The exact Bazel target shape for `rpycdec` is undecided because the actual `rpycdec` source has not been inspected in this session. The grounded next step is to inspect it first, then decide whether it should be wrapped as a `py_library`, `py_binary`, external archive, or another `//third_party` target.
The AST exporter itself needs only the Python standard library plus the target launcher's bundled Ren'Py runtime. The broader preparation flow still needs `rpycdec` for archive extraction and decompiled debug source unless that responsibility changes.
## Recommended next session start
1. Do not continue from the discarded PR implementation.
2. Inspect the actual `rpycdec` source and decide how to expose it under `//third_party/python` or another `//third_party` package.
3. Design a standalone preparation Python tool with no unnecessary external dependencies.
4. Keep the normalized AST JSON boundary between bundled Ren'Py runtime code and Rust Ariadne core.
5. Return to the Rust `State` shape after the preparation artifact contract is clearer.