docs: update SteamCMD server workflow

docs/00 INBOX/ai/steamcmd servers.md

@@ -1,28 +1,73 @@
 # SteamCMD servers
 
-Use `modules.services.steamcmd` as the low-level runner for one SteamCMD server.
+This note records the workflow used to add the Valheim server.
-Create a game module under `configs/nixos/modules/services/games/` for each game.
+The workflow starts with the lowest useful layer.
+Each later step moves game knowledge into a narrower module.
 
-The game module owns game-specific defaults and command-line flags.
+## Stage 1: Run one server with `steamcmd`
-The host owns deployment-specific paths, names, secrets, and network placement.
 
-## Create a game module
+Start by proving the generic SteamCMD runner works on a host.
+Configure `modules.services.steamcmd` directly in the host file.
 
-1. Create `configs/nixos/modules/services/games/<game>.nix`.
+```nix
-2. Define options under `modules.services.games.<game>`.
+modules.services.steamcmd = {
-3. Expose only values the host must choose.
+  enable = true;
-4. Map those values to `modules.services.steamcmd`.
+  steamDir = "/pikachu/data/steamcmd";
-5. Set the Steam dedicated-server app ID in the game module.
+  serviceName = "valheim";
-6. Set the server executable in the game module.
+  appId = 896660;
-7. Generate game-specific arguments in the game module.
+  installDir = "/pikachu/data/servers/valheim/main";
-8. Open the game's documented UDP ports in the game module.
+  stateDir = "/pikachu/media/games/servers/valheim/main";
+  executable = "./valheim_server.x86_64";
+  arguments = [
+    "-nographics"
+    "-batchmode"
+    "-name"
+    "main"
+    "-port"
+    "2456"
+    "-world"
+    "main"
+    "-savedir"
+    config.modules.services.steamcmd.stateDir
+    "-password"
+    "foobarbaz"
+    "-public"
+    "0"
+  ];
+  udpPorts = [
+    2456
+    2457
+  ];
+};
+```
 
-Keep `modules.services.steamcmd` focused on shared SteamCMD behavior.
+This layer owns SteamCMD update behavior and systemd wiring.
-Do not put game-specific flags or port rules there.
+It also owns directory setup for SteamCMD metadata, server files, and state.
 
-## Configure a host
+Do not pin the game server version in Nix.
+SteamCMD mutates the installed server files.
 
-During validation, enable the game module directly on a host.
+## Stage 2: Add a game service module
+
+After direct SteamCMD deployment works, create a game module.
+Use `configs/nixos/modules/services/games/<game>.nix`.
+
+The game module owns game-specific details:
+
+- Steam app ID
+- server executable
+- required command-line flags
+- documented UDP ports
+- private or public listing policy
+
+The host keeps deployment-specific values:
+
+- host paths
+- server name
+- secret choice
+- guest placement, when a guest exists
+
+For Valheim, the host config became:
 
 ```nix
 modules.services.games.valheim = {
@@ -35,37 +80,212 @@ modules.services.games.valheim = {
 };
 ```
 
-Use host paths for direct host deployment.
+The Valheim module translated that into `modules.services.steamcmd`.
-Use guest paths after the server moves into a guest.
+It generated the Valheim argument list and UDP ports.
 
-## Deploy and test
+## Stage 3: Add UDP ingress
 
-1. Evaluate the host configuration.
+Valheim uses UDP.
+HTTP virtual hosts do not route Valheim traffic.
 
-   ```sh
+Extend `registry.route` with UDP ports:
-   nix eval .#nixosConfigurations.<host>.config.system.build.toplevel.drvPath --raw
-   ```
 
-2. Deploy the host.
+```nix
+registry.route.pikachu.valheim = {
+  domain = "valheim.alexion.dev";
+  udp.ports = [
+    2456
+    2457
+  ];
+};
+```
 
-   ```sh
+Keep `domain` mandatory.
-   deploy-to <host>
+Nix does not own public DNS today.
-   ```
+The route domain still identifies the public route.
+A future LAN DNS module can also consume it.
 
-3. Check the game service.
+The reverse-proxy module joins `registry.route` with `registry.address`.
+It renders nginx `stream` entries for UDP ports.
 
-   ```sh
+```nginx
-   sudo systemctl status <game>.service
+server {
-   sudo journalctl -u <game>.service -n 200 --no-pager
+  listen 2456 udp;
-   ```
+  proxy_pass 10.23.20.197:2456;
+}
+```
 
-4. Check UDP listeners.
+Configure public DNS outside Nix:
 
-   ```sh
+```text
-   sudo ss -lunp | grep -E '<port>|<port-plus-one>|<game>'
+valheim.alexion.dev -> public IP
-   ```
+```
 
-5. Connect from the game client.
+Configure LAN DNS outside Nix:
 
-The Valheim server is ready after the log contains `Game server connected`.
+```text
-It can spend time generating a new world before it accepts connections.
+valheim.alexion.dev -> reverse-proxy guest IP
+```
+
+Configure router forwarding outside Nix:
+
+```text
+2456/udp -> reverse-proxy guest:2456
+2457/udp -> reverse-proxy guest:2457
+```
+
+## Stage 4: Move the server into a guest
+
+After the host-level game module works, add a guest module.
+Use `configs/nixos/guests/<game>.nix`.
+
+The guest module wraps `modules.services.games.<game>`.
+It also maps host paths into guest paths.
+
+For Valheim, the host config became:
+
+```nix
+guests.valheim = {
+  enable = true;
+  vlan = 20;
+  address = "10.23.20.197";
+  steamPath = "/pikachu/data/steamcmd";
+  installPath = "/pikachu/data/servers/valheim/main";
+  statePath = "/pikachu/media/games/servers/valheim/main";
+  serverName = "main";
+  reverseProxy = {
+    enable = true;
+    route = {
+      domain = "valheim.alexion.dev";
+      udp.ports = [
+        config.guests.valheim.port
+        (config.guests.valheim.port + 1)
+      ];
+    };
+  };
+};
+```
+
+The generic guest machinery publishes registry entries.
+
+```nix
+guests.valheim.address = "10.23.20.197";
+```
+
+becomes:
+
+```nix
+registry.address.pikachu.valheim = "10.23.20.197";
+```
+
+The guest route becomes:
+
+```nix
+registry.route.pikachu.valheim = config.guests.valheim.reverseProxy.route;
+```
+
+## Secrets
+
+Use SOPS for the game server password.
+Use one shared secret for game servers:
+
+```yaml
+services:
+  game-servers:
+    password: <password>
+```
+
+The Valheim guest defaults to:
+
+```nix
+passwordSecret = "services/game-servers/password";
+```
+
+The host decrypts the secret.
+The guest module bind-mounts the decrypted file into the guest.
+The Valheim module reads the password file at runtime.
+
+Edit shared secrets with the host key:
+
+```sh
+cd configs/nixos
+sops-host secrets/shared.yaml
+```
+
+## Validation
+
+Evaluate the host:
+
+```sh
+nix eval .#nixosConfigurations.pikachu.config.system.build.toplevel.drvPath --raw
+```
+
+Deploy the host:
+
+```sh
+deploy-to pikachu
+```
+
+Check the host secret:
+
+```sh
+sudo test -s /run/secrets/services/game-servers/password && echo secret-ok
+```
+
+Check the Valheim guest:
+
+```sh
+sudo systemctl status container@valheim.service --no-pager
+```
+
+Check the guest secret:
+
+```sh
+sudo nixos-container run valheim -- test -s /run/secrets/services/game-servers/password
+```
+
+Check the Valheim service:
+
+```sh
+sudo nixos-container run valheim -- systemctl status valheim.service --no-pager
+sudo nixos-container run valheim -- journalctl -u valheim.service -n 200 --no-pager
+```
+
+Wait for these log lines:
+
+```text
+Game server connected
+Opened Steam server
+```
+
+Check Valheim UDP listeners:
+
+```sh
+sudo nixos-container run valheim -- ss -lunp | grep -E '2456|2457|valheim'
+```
+
+Check reverse-proxy UDP listeners:
+
+```sh
+sudo nixos-container run reverse-proxy -- ss -lunp | grep -E '2456|2457|nginx'
+```
+
+Test the client through LAN DNS or public DNS:
+
+```text
+valheim.alexion.dev:2456
+```
+
+Test direct guest connectivity when ingress fails:
+
+```text
+10.23.20.197:2456
+```
+
+## Operational notes
+
+A first Valheim start can spend time generating a world.
+The service accepts connections after `Opened Steam server` appears.
+
+SteamCMD can leave old ownership on migrated server files.
+The SteamCMD module uses recursive tmpfiles repair for its managed directories.