docs: add ariadne planning notes #26
17
docs/.obsidian/workspace.json
vendored
17
docs/.obsidian/workspace.json
vendored
@@ -124,7 +124,7 @@
|
||||
}
|
||||
],
|
||||
"direction": "horizontal",
|
||||
"width": 200
|
||||
"width": 338.5
|
||||
},
|
||||
"right": {
|
||||
"id": "c35c41e87bb5c18f",
|
||||
@@ -231,13 +231,17 @@
|
||||
},
|
||||
"active": "0c587e5452c16787",
|
||||
"lastOpenFiles": [
|
||||
"00 INBOX/home assistant agent integration.md",
|
||||
"00 INBOX/ai/ariadne-project-goals.md",
|
||||
"00 INBOX/ai/dictionary-cli.md",
|
||||
"00 INBOX/ai/dictionary-skill.md",
|
||||
"03 wrk/falinks.md",
|
||||
"03 wrk/ariadne.md",
|
||||
"03 wrk/machoke.md",
|
||||
"TODO.md",
|
||||
"00 INBOX/home assistant agent integration.md",
|
||||
"02 notes/falinks public surfaces.md",
|
||||
"03 wrk/machoke.md~",
|
||||
"03 wrk/machoke.md",
|
||||
"02 notes/machoke public surface.md",
|
||||
"03 wrk/falinks.md",
|
||||
"03 wrk/falinks.md~",
|
||||
"99 meta/templates/checkmark.md",
|
||||
"AGENTS.md",
|
||||
@@ -257,11 +261,6 @@
|
||||
"01 sources/obsidian and ai were made for each other.md",
|
||||
"99 meta/technical writing.md",
|
||||
"99 meta/dictionary.md",
|
||||
"03 wrk/dotfiles.md",
|
||||
"00 INBOX/ai/dotfiles flake architecture document.md",
|
||||
"00 INBOX/ai/dotfiles technical debt plan.md",
|
||||
"99 meta/templates/project.md",
|
||||
"99 meta/templates/source.md",
|
||||
"99 meta/Untitled",
|
||||
"03 wrk",
|
||||
"02 notes",
|
||||
|
||||
68
docs/00 INBOX/ai/ariadne-project-goals.md
Normal file
68
docs/00 INBOX/ai/ariadne-project-goals.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Ariadne project goals
|
||||
|
||||
Ariadne analyzes a local Ren'Py game install.
|
||||
It extracts source, builds a source graph, and plans routes through target nodes.
|
||||
|
||||
## Privacy constraint
|
||||
|
||||
Do not write the target game's title in the monorepo.
|
||||
Use `target game` or `Ren'Py game` in committed notes, code, and pull requests.
|
||||
Keep title-specific paths and names in ignored local data only.
|
||||
|
||||
## Goal order
|
||||
|
||||
1. Unlock all Steam achievements.
|
||||
2. Maximize Season 3 dialogue coverage.
|
||||
3. Extend the same coverage work to Seasons 1 and 2.
|
||||
|
||||
## Steam achievements
|
||||
|
||||
Achievement unlocks appear in source as reachable calls.
|
||||
The first planning target is every achievement grant node.
|
||||
|
||||
Ariadne needs to find each grant call and compute route conditions that reach it.
|
||||
This is source-graph coverage over sparse semantic targets.
|
||||
|
||||
## Season 3 dialogue coverage
|
||||
|
||||
Season 3 exposes an in-game percentage for seen dialogue.
|
||||
The stat computes Ren'Py dialogue-block coverage from persistent seen data.
|
||||
|
||||
For this game, dialogue blocks map mainly to unique `Say` statements.
|
||||
Ariadne should target Season 3 `Say` nodes as coverage targets.
|
||||
|
||||
The in-game percentage gives a validation signal for Ariadne's coverage model.
|
||||
Season 3 also rewards high coverage inside the game.
|
||||
|
||||
## Seasons 1 and 2 coverage
|
||||
|
||||
Seasons 1 and 2 lack the same in-game percentage display.
|
||||
Ariadne can still compute source coverage for their dialogue nodes.
|
||||
|
||||
Validation for these seasons will depend on Ariadne's own model.
|
||||
Persistent seen data can still help later.
|
||||
|
||||
## State model phases
|
||||
|
||||
The first phase ignores player state.
|
||||
It asks which routes cover all targets from source alone.
|
||||
|
||||
A later phase reads existing player state.
|
||||
It uses save files or persistent data to remove completed targets.
|
||||
|
||||
That phase produces a personalized plan for remaining achievements and dialogue coverage.
|
||||
|
||||
## Graph model
|
||||
|
||||
The source graph needs typed nodes for labels, jumps, conditions, menus, visual statements, dialogue, and achievement grants.
|
||||
Menus and conditions require branch edges and merge edges.
|
||||
|
||||
Dialogue and visual nodes support route explanation.
|
||||
Achievement and dialogue nodes become coverage targets.
|
||||
|
||||
## Near-term implementation direction
|
||||
|
||||
Keep extraction and decompilation generic.
|
||||
Parse source into typed graph artifacts.
|
||||
Add achievement grant nodes before broad route planning.
|
||||
Add Season 3 dialogue nodes as the next coverage target set.
|
||||
312
docs/03 wrk/ariadne.md
Normal file
312
docs/03 wrk/ariadne.md
Normal file
@@ -0,0 +1,312 @@
|
||||
# Ariadne design notes
|
||||
|
||||
## Purpose
|
||||
|
||||
These notes capture the design state from the Ariadne pipeline and Ren'Py AST discussion. They are intended to let a new session resume the design without relying on the discarded pull-request work.
|
||||
|
||||
Do not assume the implementation described here still exists in the Ariadne worktree. The pull request work was considered exploratory and should be dropped or redone cleanly.
|
||||
|
||||
## Core state model
|
||||
|
||||
Ariadne's core state should be graph-first. The current agreed shape is:
|
||||
|
||||
```rust
|
||||
State {
|
||||
nodes: Map<NodeKey, Node>,
|
||||
edges: Map<NodeKey, Set<NodeKey>>,
|
||||
pending: Vec<Operation>,
|
||||
}
|
||||
```
|
||||
|
||||
Meaning:
|
||||
|
||||
- `nodes` is the committed node store.
|
||||
- `edges` is committed directed connectivity.
|
||||
- `pending` is a first-class operation log waiting to be applied.
|
||||
|
||||
The mutation vocabulary is expected to be an enum along these lines:
|
||||
|
||||
```rust
|
||||
enum Operation {
|
||||
AddNode { key: NodeKey, node: Node },
|
||||
EditNode { key: NodeKey, patch: NodePatch },
|
||||
RemoveNode { key: NodeKey },
|
||||
AddEdge { from: NodeKey, to: NodeKey },
|
||||
RemoveEdge { from: NodeKey, to: NodeKey },
|
||||
}
|
||||
```
|
||||
|
||||
Stages should not directly mutate committed state. A stage should inspect the current state and emit operations. Applying pending operations advances the graph.
|
||||
|
||||
## Edges
|
||||
|
||||
Edges are simple directed connectivity for now:
|
||||
|
||||
```text
|
||||
from -> to
|
||||
```
|
||||
|
||||
A node may have multiple outgoing edges, so a literal `Map<NodeKey, NodeKey>` is not enough. Use a shape equivalent to:
|
||||
|
||||
```rust
|
||||
Map<NodeKey, Set<NodeKey>>
|
||||
```
|
||||
|
||||
Do not put conditional meaning on edges yet. The Python prototype solved conditional flow structurally:
|
||||
|
||||
- conditional expressions live on conditional nodes;
|
||||
- menu choice conditions live on menu-choice or menu data;
|
||||
- branch and merge edges are ordinary directed edges.
|
||||
|
||||
This means edge metadata is not currently required. If a future requirement needs multiple semantic edges between the same pair of nodes, revisit the edge model then.
|
||||
|
||||
## Node identity
|
||||
|
||||
Label names are not valid primary node keys. We checked combined source extracted from multiple launchers and found collisions, including `label start:` appearing in more than one source universe. There were dozens of duplicate label names across the combined prepared source.
|
||||
|
||||
The current direction is that `NodeKey` should include a namespace:
|
||||
|
||||
```rust
|
||||
NodeKey {
|
||||
namespace,
|
||||
path,
|
||||
line,
|
||||
occurrence,
|
||||
}
|
||||
```
|
||||
|
||||
The namespace identifies the launcher/source universe. Edges may cross namespaces, because later launchers can depend on decisions or state from earlier launchers.
|
||||
|
||||
`path` and `line` identify the source location inside the namespace. `occurrence` became relevant after inspecting Ren'Py AST output: one physical source line can produce multiple AST nodes, for example display/transition nodes around the same line. If we use AST-derived nodes, line alone is not enough.
|
||||
|
||||
Earlier alternatives that were rejected or deferred:
|
||||
|
||||
- label-name keys, because labels collide across source universes;
|
||||
- flat string keys without structure, because stages will need to reason about namespace and source identity;
|
||||
- columns, because column-level identity is not currently useful;
|
||||
- ranges, because the Python prototype did not model ranges, and multiline statements were rare in that prototype.
|
||||
|
||||
Multiline statements do exist. The Python prototype found a very small number and chose to treat them as physical-line parser limitations. For Ariadne, if AST ingestion is used, Ren'Py's own AST should handle logical statements better than our source parser.
|
||||
|
||||
## Dialogue blocks
|
||||
|
||||
A dialogue block is the Ren'Py/game unit counted by seen-dialogue coverage. For Ariadne, dialogue-block coverage maps mainly to unique Ren'Py `Say` statements.
|
||||
|
||||
Important identity rule:
|
||||
|
||||
```text
|
||||
same dialogue text at different source locations = different dialogue blocks
|
||||
```
|
||||
|
||||
So dialogue-block identity must include source identity. It must not be based only on speaker and text.
|
||||
|
||||
## Python prototype lessons
|
||||
|
||||
The previous Python prototype used `metadata.key` strings on parsed nodes. Its key assignment was approximately:
|
||||
|
||||
```python
|
||||
if node is LabelNode:
|
||||
key = node.name
|
||||
elif node is InitNode or InitPriorityOffsetNode:
|
||||
key = f"{path.stem}:init:{line_no}"
|
||||
elif node has parent:
|
||||
key = f"{parent.key}:{line_no}"
|
||||
else:
|
||||
key = str(line_no)
|
||||
```
|
||||
|
||||
This was useful for a narrower source universe but is not enough for Ariadne's multi-launcher model.
|
||||
|
||||
The Python prototype also solved conditional and menu edges with structural graph logic:
|
||||
|
||||
- each label/init owns a subgraph;
|
||||
- body statements are chained sequentially;
|
||||
- conditionals use a stack of branch frames;
|
||||
- branch tails merge into the closing successor;
|
||||
- jump-terminated branches do not get false fallthrough merge edges;
|
||||
- menus fan out to choices and fan back into a shared continuation.
|
||||
|
||||
Those ideas should inform future edge-building stages.
|
||||
|
||||
## Ren'Py AST ingestion decision
|
||||
|
||||
We investigated using Ren'Py AST as input instead of parsing decompiled `.rpy` source.
|
||||
|
||||
Direct Rust `.rpyc` pickle decoding partially worked:
|
||||
|
||||
```text
|
||||
.rpyc -> RPYC slot table -> zlib -> pickle -> AST-like values
|
||||
```
|
||||
|
||||
It succeeded on small/simple files, but failed on branch-heavy files with recursive pickle/object structure. The conclusion is not to make raw Ren'Py pickle decoding the immediate Rust path.
|
||||
|
||||
Preferred approach:
|
||||
|
||||
```text
|
||||
launcher-bundled Python/Ren'Py runtime
|
||||
-> load .rpyc AST objects
|
||||
-> export normalized Ariadne-neutral JSON records
|
||||
-> Rust imports JSON into State operations
|
||||
```
|
||||
|
||||
Ren'Py AST is an input format, not Ariadne's internal model.
|
||||
|
||||
## Ren'Py AST object shape
|
||||
|
||||
Ren'Py AST objects are Python objects from `renpy.ast`, with one class per statement-like construct. Common base fields include:
|
||||
|
||||
```python
|
||||
node.filename
|
||||
node.linenumber
|
||||
node.name
|
||||
node.next
|
||||
```
|
||||
|
||||
Examples of useful node classes and fields:
|
||||
|
||||
```python
|
||||
Label:
|
||||
name
|
||||
block
|
||||
parameters
|
||||
hide
|
||||
|
||||
Say:
|
||||
who
|
||||
what
|
||||
identifier
|
||||
attributes
|
||||
temporary_attributes
|
||||
arguments
|
||||
with_
|
||||
|
||||
If:
|
||||
entries # list of (condition, block)
|
||||
|
||||
Menu:
|
||||
items # list of (label_text, condition, block)
|
||||
|
||||
Jump:
|
||||
target
|
||||
expression
|
||||
|
||||
Call:
|
||||
label
|
||||
expression
|
||||
arguments
|
||||
|
||||
Python:
|
||||
code.source
|
||||
store
|
||||
hide
|
||||
```
|
||||
|
||||
For dialogue coverage, `Say.identifier` and translation wrapper information may be important. Slot choice matters: slot 1 has the pre-static-transform AST; slot 2 can include translation wrappers such as `Translate` and `EndTranslate` around `Say` nodes.
|
||||
|
||||
## Prototype that was created and should be discarded
|
||||
|
||||
A prototype was created in the Ariadne worktree but should not be treated as the desired final shape:
|
||||
|
||||
```text
|
||||
src/ariadne/prototypes/renpy-ast-adapter/
|
||||
src/ariadne/tools/export-renpy-ast.py
|
||||
```
|
||||
|
||||
It tested:
|
||||
|
||||
```text
|
||||
.rpyc slot
|
||||
-> bundled Python adapter
|
||||
-> normalized AST JSON records
|
||||
-> Rust State { nodes, edges, pending }
|
||||
```
|
||||
|
||||
The normalized JSON record shape was:
|
||||
|
||||
```json
|
||||
{
|
||||
"key": {
|
||||
"namespace": "season-3",
|
||||
"path": "game/scripts/.../file.rpy",
|
||||
"line": 123,
|
||||
"occurrence": 0
|
||||
},
|
||||
"kind": "Say",
|
||||
"fields": {
|
||||
"who": "mc",
|
||||
"what": "...",
|
||||
"identifier": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The prototype successfully exported a branch-heavy `.rpyc` file that direct Rust pickle decoding could not handle. It showed that AST ingestion can produce thousands of nodes and hundreds of repeated source locations.
|
||||
|
||||
Why discard/rework it:
|
||||
|
||||
- it was added into the existing shell preparation script too quickly;
|
||||
- the shell script became responsible for too much;
|
||||
- Bazel runfiles handling made the shell script messy;
|
||||
- the PR should be refocused before this work continues.
|
||||
|
||||
## Preparation output direction
|
||||
|
||||
Preparation output should evolve from:
|
||||
|
||||
```text
|
||||
output/
|
||||
source/
|
||||
work/
|
||||
logs/
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```text
|
||||
output/
|
||||
ast/ # normalized Ren'Py AST JSON
|
||||
source/ # decompiled/debug source
|
||||
work/ # extracted raw/intermediate files
|
||||
logs/
|
||||
```
|
||||
|
||||
Decompiled source remains useful as a debug artifact, but may stop being the primary parser input.
|
||||
|
||||
## Preparation tooling direction
|
||||
|
||||
The shell script is becoming too broad. A better next step is to move preparation into its own Python tool before returning to Ariadne core.
|
||||
|
||||
That preparation tool should own:
|
||||
|
||||
- game-root and launcher discovery;
|
||||
- archive extraction;
|
||||
- `.rpyc` collection;
|
||||
- AST export through the matching bundled Ren'Py runtime;
|
||||
- optional decompilation for debug source;
|
||||
- output layout and logs.
|
||||
|
||||
Ariadne core should only depend on the preparation tool's normalized artifacts.
|
||||
|
||||
## Python dependencies and vendoring
|
||||
|
||||
For this repository, "vendor" means making a dependency available behind a public `//third_party/...` target, like Rust crates are exposed through `//third_party/crates`.
|
||||
|
||||
If the preparation tool depends on `rpycdec`, Ariadne code should depend on a public alias such as:
|
||||
|
||||
```text
|
||||
//third_party/python:rpycdec
|
||||
```
|
||||
|
||||
not directly on the underlying source, pip repository, or external workspace.
|
||||
|
||||
The exact Bazel target shape for `rpycdec` is undecided because the actual `rpycdec` source has not been inspected in this session. The grounded next step is to inspect it first, then decide whether it should be wrapped as a `py_library`, `py_binary`, external archive, or another `//third_party` target.
|
||||
|
||||
The AST exporter itself needs only the Python standard library plus the target launcher's bundled Ren'Py runtime. The broader preparation flow still needs `rpycdec` for archive extraction and decompiled debug source unless that responsibility changes.
|
||||
|
||||
## Recommended next session start
|
||||
|
||||
1. Do not continue from the discarded PR implementation.
|
||||
2. Inspect the actual `rpycdec` source and decide how to expose it under `//third_party/python` or another `//third_party` package.
|
||||
3. Design a standalone preparation Python tool with no unnecessary external dependencies.
|
||||
4. Keep the normalized AST JSON boundary between bundled Ren'Py runtime code and Rust Ariadne core.
|
||||
5. Return to the Rust `State` shape after the preparation artifact contract is clearer.
|
||||
@@ -24,7 +24,6 @@
|
||||
- skills
|
||||
|
||||
# tech debt
|
||||
- git worktrees for monorepo
|
||||
- refactor/review `configs/nixos`
|
||||
|
||||
# learning
|
||||
|
||||
Reference in New Issue
Block a user