dotfiles/.claude/spec/guests.md

Guests

Problem Statement

The user runs a homelab on Proxmox with LXC system containers and Podman, replacing it with a fully declarative NixOS configuration where both the machines and the services they host are NixOS, built from this one flake. Each service today is an isolated guest that is a first-class citizen on a VLAN-tagged LAN: it has its own MAC address and its own IP on a specific tagged VLAN, and it writes to shared ZFS pools without permission errors. The flake currently has only two concepts, the Host and the Module, and a Module can only be turned on at host level. There is no way to express "run this service isolated in its own guest, on this VLAN, with these pool mounts" declaratively, and no way to reuse such a definition across machines. The Proxmox setup the user is leaving also imposes an imperative container lifecycle and mutable state that drifts outside version control, which is the thing the migration exists to eliminate.

Solution

Introduce a third concept alongside Host and Module: the Guest. A Guest is a reusable, machine-independent definition that bundles the Modules it runs inside an isolated NixOS instance, realized on a Host as a nested container. A Host composes both Modules and Guests and reads as one flat checklist, where a Module turned on directly is a host-level install and a Guest is a service running in its own isolated instance. Each Guest becomes a first-class L2 citizen on a chosen tagged VLAN with its own MAC and its own IP, writes to shared pools through a common storage group with no permission juggling, and carries its own resource caps and secrets. Every Guest ships with the user's baseline interactive toolset and SSH access, so any guest the user shells into is immediately a workable environment. A Guest can also run Podman or other OCI containers in its interior, which is the declarative home for image-only software that is not worth reimplementing in Nix. The foundations a Guest stands on — declarative VLAN and bridge networking, ZFS pool import, and a shared write convention — are themselves host-level Modules, so a Host declares its trunk, its VLANs, and its pools once and every Guest attaches to them.

User Stories

1. As the operator, I want to define a service as a Guest in its own file, so that the service and everything it runs are one reusable, version-controlled fact.
2. As the operator, I want a Guest to follow the same file, folder, and namespace conventions as a Module, so that I never learn a second layout and a Guest's location is its namespace.
3. As the operator, I want a Host to turn a Guest on with enable exactly as it turns a Module on, so that a Host stays a single flat checklist of everything it carries.
4. As the operator, I want a Module placed directly on a Host to mean a host-level install and a Guest to mean an isolated instance, so that the same checklist expresses both placements without ambiguity.
5. As the operator, I want a Guest to give its service its own MAC and its own IP on a specific tagged VLAN, so that my VLAN-segmented network treats each service as a distinct L2 citizen, exactly as Proxmox did.
6. As the operator, I want to reuse the MAC addresses my existing containers already use, so that my router's DHCP reservations keep working and the migration needs no network reconfiguration.
7. As the operator, I want a Guest whose MAC I did not set to still get a stable, readable MAC, so that I can add a reservation for a new guest without hand-assigning addresses.
8. As the operator, I want a Guest to take its address by DHCP by default and optionally a static address, so that IP management stays centralized at my router where it already lives.
9. As the operator, I want a build-time error when a Guest names a VLAN its Host has not declared, so that a misplacement fails at evaluation rather than as a broken bridge at runtime.
10. As the operator, I want a Guest to mount shared pool paths at any granularity, a single folder or a whole pool, read-only or read-write, so that each service sees exactly the data it should.
11. As the operator, I want every Guest's service to write to shared pools without permission errors, so that I never repeat the Proxmox idmap dance.
12. As the operator, I want a Guest to receive only the decrypted secrets it names, so that services get their credentials while no guest ever holds a decryption key.
13. As the operator, I want to cap a Guest's memory, CPU, and process count, so that one misbehaving service cannot starve its Host.
14. As the operator, I want every Guest to come with fish, tmux, nvim, git, and direnv, so that any guest I shell into feels like my own machine.
15. As the operator, I want to reach a Guest both directly over SSH with my keys and from its Host with machinectl, so that I always have a way in whether or not the network path is open.
16. As the operator, I want a Guest to run Podman and OCI containers in its interior by turning on a nesting capability, so that image-only software has a declarative home without a separate mechanism.
17. As the operator, I want the same Guest definition to be deployable on more than one Host by declaring it there with that Host's placement, so that an appliance is portable between machines.
18. As the operator, I want the networking, storage, and write conventions to be host-level Modules I declare once per Host, so that Guests attach to shared foundations instead of each re-specifying the machine.
19. As the operator, I want a Guest's innards fixed in the Guest file and only its placement supplied by the Host, so that a Guest behaves identically wherever it runs and reads honestly in isolation.
20. As the operator, I want to build a Host and know its Guests evaluate and its placements are consistent before I deploy, so that a rebuild is trustworthy.

Implementation Decisions

The Guest concept

• A Guest is a new, auto-loaded kind of definition, resolved in CONTEXT.md and formalized in ADR 0006. The Auto-loader discovers every Guest under guests/ as a third kind alongside Modules and Hosts.
• A Guest follows the Namespace convention and the file-or-folder rule of a Module verbatim: a plain Guest is one file whose location is its namespace, and a Guest that needs auxiliary files becomes a folder, with subfolders as namespace segments and the same index-node rule (per ADR 0004).
• A Guest is Module-shaped: it declares its own guests.<path> option namespace carrying an enable plus its placement fields, and guards its body on that enable following the Enable convention. The one difference from a Module is the payload: a Module's body merges settings into the Host, whereas a Guest's body realizes a nested container running the Guest's interior.
• A Guest is backend-agnostic through a backend field, but only the container backend (systemd-nspawn, via the platform's native nested-container mechanism) is built. microvm is a reserved backend value that is not implemented in this work.
• A Guest is sealed and singleton: the Modules a Guest runs are fixed in the Guest file and are not overridable by a Host, and a Guest is instantiated at most once per Host, keyed by its namespace path. Running more than one instance of a service on a Host is out of scope for this work.

Placement interface (Host-side)

A Host instantiates a Guest by setting fields under guests.<path>. The Guest owns its interior Modules; the Host owns only this placement.

Field	Meaning	Default
enable	Turn the Guest on for this Host	off
backend	Realization backend	container
vlan	Tagged VLAN the Guest lives on; maps to the Host's br-vlan<id> bridge by convention	required when networked
mac	The Guest's MAC address; set it to reuse an existing address	derived deterministically and surfaced when unset
address	Static address on the VLAN	unset, meaning DHCP
mounts	Map of guest path to host path, each with a per-mount readOnly	read-write per mount
secrets	Names of secrets the Guest needs	none
limits	memory, cpu, tasksMax caps applied to the guest's unit	uncapped
nesting	Grant the nested-container prerequisites so the interior can run Podman/OCI	off
autoStart	Start the Guest at boot	on

• The vlan field maps to a bridge by the br-vlan<id> naming convention, so a Guest states only which VLAN it lives on.
• A build-time assertion ties a Guest's vlan to the set of VLANs its Host's network foundation declares, so a Guest on an undeclared VLAN fails the Host build with a clear message.
• The mac field is pinned inside the guest through the guest's own systemd-networkd, which is the only way a nested-container MAC is stable; an unset mac derives a stable address from the Guest's namespace path in a locally-administered range, readable via evaluation so the operator can add a reservation.
• The mounts field is realized as the nested container's bind mounts and defaults to read-write, matching the migration reality that services must write to pools.
• The nesting field is what makes the OCI fallback a plain Guest: the Skeleton emits the nested-container cgroup-delegation and capability prerequisites once, so the operator flips one boolean and the interior's oci-containers runtime works, with Podman as the default runtime.

Bases and the shared environment

• The shared base config splits in two (recorded under the Skeleton term): a host base that carries host-only machinery (bootloader, hardware profile, host identity, boot and garbage-collection timers), and a slim guest-base that every nested Guest stands on.
• Every Guest imports the full modules/ tree, so any Module is available to enable inside a Guest; only Modules whose needs the base meets are enabled in practice.
• A Guest gets home-manager and the same primary user as a Host, which makes every Module placement-agnostic and removes any need for a host-versus-guest Module taxonomy.
• The guest-base auto-enables the toolkit bundle and modules.ssh, so every Guest has the baseline toolset and SSH access without per-Guest wiring. Reaching a Guest by machinectl from its Host needs no Guest configuration and is the always-available fallback.

New and modified Modules

• modules.toolkit (new): a deliberate bundle Module turning on fish, tmux, nvim, git, and direnv as one unit. The guest-base auto-enables it; a Host enables it explicitly, keeping the Host a full checklist. This is a bundle wanted as a unit, distinct from the grouping-directory enables ADR 0004 rejected.
• modules.network (new): the host-level networking foundation. A Host declares its trunk interface and the set of VLANs to materialize, and the Module emits one bridge per tagged VLAN with systemd-networkd and manages the Host's own management address.
• modules.zfs (new): the host-level pool import. A Host declares its host id, the pools to import, and their dataset mountpoints; the pools are durable state that is imported, never rebuilt.
• A shared storage group with a fixed gid in the shared base gives 1:1 ownership between Host and Guest. Because the container backend uses identity mapping, a guest service that writes as the storage group lands on the pool as that same group, which is the entire "no permission errors" mechanism.
• modules.ssh (reused): the guest-base enables it in a guest flavor, with the operator's authorized keys and a self-generated host key, since a Guest does not carry a per-Guest host identity the way a Host does.
• The Skeleton grows the Guest realization: the Auto-loader's discovery of guests/, the compilation of a Guest's interior into a nested container, and the wiring of placement fields (VLAN attachment, MAC pinning, bind mounts, secret mounts, unit caps, nesting prerequisites).

Secrets

• The Host is the sole decryptor, consistent with the host identity of ADR 0002 and the existing sops-nix setup. The Host decrypts, and a Guest receives only the specific secret files it names, read-only bind-mounted in, with ownership aligned by the identity mapping. A Guest holds no age key.

Testing Decisions

A good test here exercises external, observable behavior of a Guest and its foundations, not the internal shape of the generated nested-container config. The load-bearing behaviors are: a Guest presents its own MAC and its own IP on the correct tagged VLAN, a guest service writes to a shared-group mount without permission error, the baseline toolset and SSH access are present, and a misplaced Guest fails the build.

Two seams, the fewest that cover the work:

1. The Host toplevel build via nix flake check — the existing, primary seam every Host already has. This is where a Guest's evaluation, its placement fields, the base split, the Auto-loader wiring, and the vlan-against-declared-VLANs assertion are all verified. Cheap targeted evaluations of derived values (a Guest's resolved bridge attachment, its derived MAC, the shared storage gid) ride on this same seam.
2. One NixOS VM integration test, added to the flake's checks so nix flake check runs it — a new seam, and the highest behavioral one available. A single harness boots the modules.network foundation and one sample Guest and asserts the three hard requirements together: the guest has its own MAC, gets its own IP on a tagged VLAN across a virtual L2 segment, and can write to a bind-mounted directory owned by the shared storage group.

Prior art: the flake's checks.<host> toplevel builds are the established evaluation seam; the upstream NixOS test suite exercises nested-container networking, including macvlan and extra-veth cases, and is the model for the VM integration test. The two behaviors a VM cannot honestly reproduce — real 802.1Q against the physical switch and real ZFS identity-mapped writes on the pool — are verified manually on the target Host rather than in the test.

Out of Scope

• The microvm backend and any hard-isolation guest; the concept reserves the backend value but this work does not build it.
• Running more than one instance of a service on a Host; the Guest stays singleton and the operator will address multiplicity separately.
• Exotic OCI images that need their own nested init or unusual storage drivers, which are the future reason to reach for the microvm backend.
• The concrete homelab Host itself and its real values — its trunk interface name, VLAN ids, pool names, and per-Guest MAC and mount assignments — which are host-specific data supplied when a Host is added.
• Migrating specific services (the *arr stack, download clients, media servers) into Guests; this work delivers the concept and its foundations, not the service catalogue.
• Real-switch VLAN behavior and real ZFS identity-mapped writes, which are verified manually on the target Host.

Further Notes

• The decision to introduce Guest as a third concept, backend-agnostic but container-only for now, sealed and singleton, is recorded in ADR 0006.
• The "no permission errors" result rests on the container backend's identity mapping being the privileged-container equivalent the operator already trusts from Proxmox; it is deliberately not the unprivileged idmap model, which is what made pool writes painful before.
• Pools are durable state imported by the Host, never recreated by a rebuild, so a service's data survives any rebuild or reimage.
• Podman is the default and recommended runtime inside a nesting Guest, matching what the operator already runs; a container-backend Guest is soft-isolated (shared kernel), the same isolation class as the Proxmox LXCs being replaced, so nothing is lost on that axis in the move.