chore(agents): remove obsolete guest scaffolding
This commit is contained in:
@@ -1,140 +0,0 @@
|
|||||||
# Guests
|
|
||||||
|
|
||||||
## Problem Statement
|
|
||||||
|
|
||||||
The user runs a homelab on Proxmox with LXC system containers and Podman, replacing it with a fully declarative NixOS configuration where both the machines and the services they host are NixOS, built from this one flake.
|
|
||||||
Each service today is an isolated guest that is a first-class citizen on a VLAN-tagged LAN: it has its own MAC address and its own IP on a specific tagged VLAN, and it writes to shared ZFS pools without permission errors.
|
|
||||||
The flake currently has only two concepts, the Host and the Module, and a Module can only be turned on at host level.
|
|
||||||
There is no way to express "run this service isolated in its own guest, on this VLAN, with these pool mounts" declaratively, and no way to reuse such a definition across machines.
|
|
||||||
The Proxmox setup the user is leaving also imposes an imperative container lifecycle and mutable state that drifts outside version control, which is the thing the migration exists to eliminate.
|
|
||||||
|
|
||||||
## Solution
|
|
||||||
|
|
||||||
Introduce a third concept alongside Host and Module: the Guest.
|
|
||||||
A Guest is a reusable, machine-independent definition that bundles the Modules it runs inside an isolated NixOS instance, realized on a Host as a nested container.
|
|
||||||
A Host composes both Modules and Guests and reads as one flat checklist, where a Module turned on directly is a host-level install and a Guest is a service running in its own isolated instance.
|
|
||||||
Each Guest becomes a first-class L2 citizen on a chosen tagged VLAN with its own MAC and its own IP, writes to shared pools through a common storage group with no permission juggling, and carries its own resource caps and secrets.
|
|
||||||
Every Guest ships with the user's baseline interactive toolset and SSH access, so any guest the user shells into is immediately a workable environment.
|
|
||||||
A Guest can also run Podman or other OCI containers in its interior, which is the declarative home for image-only software that is not worth reimplementing in Nix.
|
|
||||||
The foundations a Guest stands on — declarative VLAN and bridge networking, ZFS pool import, and a shared write convention — are themselves host-level Modules, so a Host declares its trunk, its VLANs, and its pools once and every Guest attaches to them.
|
|
||||||
|
|
||||||
## User Stories
|
|
||||||
|
|
||||||
1. As the operator, I want to define a service as a Guest in its own file, so that the service and everything it runs are one reusable, version-controlled fact.
|
|
||||||
2. As the operator, I want a Guest to follow the same file, folder, and namespace conventions as a Module, so that I never learn a second layout and a Guest's location is its namespace.
|
|
||||||
3. As the operator, I want a Host to turn a Guest on with `enable` exactly as it turns a Module on, so that a Host stays a single flat checklist of everything it carries.
|
|
||||||
4. As the operator, I want a Module placed directly on a Host to mean a host-level install and a Guest to mean an isolated instance, so that the same checklist expresses both placements without ambiguity.
|
|
||||||
5. As the operator, I want a Guest to give its service its own MAC and its own IP on a specific tagged VLAN, so that my VLAN-segmented network treats each service as a distinct L2 citizen, exactly as Proxmox did.
|
|
||||||
6. As the operator, I want to reuse the MAC addresses my existing containers already use, so that my router's DHCP reservations keep working and the migration needs no network reconfiguration.
|
|
||||||
7. As the operator, I want a Guest whose MAC I did not set to still get a stable, readable MAC, so that I can add a reservation for a new guest without hand-assigning addresses.
|
|
||||||
8. As the operator, I want a Guest to take its address by DHCP by default and optionally a static address, so that IP management stays centralized at my router where it already lives.
|
|
||||||
9. As the operator, I want a build-time error when a Guest names a VLAN its Host has not declared, so that a misplacement fails at evaluation rather than as a broken bridge at runtime.
|
|
||||||
10. As the operator, I want a Guest to mount shared pool paths at any granularity, a single folder or a whole pool, read-only or read-write, so that each service sees exactly the data it should.
|
|
||||||
11. As the operator, I want every Guest's service to write to shared pools without permission errors, so that I never repeat the Proxmox idmap dance.
|
|
||||||
12. As the operator, I want a Guest to receive only the decrypted secrets it names, so that services get their credentials while no guest ever holds a decryption key.
|
|
||||||
13. As the operator, I want to cap a Guest's memory, CPU, and process count, so that one misbehaving service cannot starve its Host.
|
|
||||||
14. As the operator, I want every Guest to come with fish, tmux, nvim, git, and direnv, so that any guest I shell into feels like my own machine.
|
|
||||||
15. As the operator, I want to reach a Guest both directly over SSH with my keys and from its Host with `machinectl`, so that I always have a way in whether or not the network path is open.
|
|
||||||
16. As the operator, I want a Guest to run Podman and OCI containers in its interior by turning on a nesting capability, so that image-only software has a declarative home without a separate mechanism.
|
|
||||||
17. As the operator, I want the same Guest definition to be deployable on more than one Host by declaring it there with that Host's placement, so that an appliance is portable between machines.
|
|
||||||
18. As the operator, I want the networking, storage, and write conventions to be host-level Modules I declare once per Host, so that Guests attach to shared foundations instead of each re-specifying the machine.
|
|
||||||
19. As the operator, I want a Guest's innards fixed in the Guest file and only its placement supplied by the Host, so that a Guest behaves identically wherever it runs and reads honestly in isolation.
|
|
||||||
20. As the operator, I want to build a Host and know its Guests evaluate and its placements are consistent before I deploy, so that a rebuild is trustworthy.
|
|
||||||
|
|
||||||
## Implementation Decisions
|
|
||||||
|
|
||||||
### The Guest concept
|
|
||||||
|
|
||||||
- A **Guest** is a new, auto-loaded kind of definition, resolved in `CONTEXT.md` and formalized in ADR 0006.
|
|
||||||
The Auto-loader discovers every Guest under `guests/` as a third kind alongside Modules and Hosts.
|
|
||||||
- A Guest follows the **Namespace convention** and the file-or-folder rule of a Module verbatim: a plain Guest is one file whose location is its namespace, and a Guest that needs auxiliary files becomes a folder, with subfolders as namespace segments and the same index-node rule (per ADR 0004).
|
|
||||||
- A Guest is **Module-shaped**: it declares its own `guests.<path>` option namespace carrying an `enable` plus its placement fields, and guards its body on that `enable` following the **Enable convention**.
|
|
||||||
The one difference from a Module is the payload: a Module's body merges settings into the Host, whereas a Guest's body **realizes a nested container** running the Guest's interior.
|
|
||||||
- A Guest is **backend-agnostic** through a `backend` field, but only the `container` backend (systemd-nspawn, via the platform's native nested-container mechanism) is built.
|
|
||||||
`microvm` is a reserved backend value that is not implemented in this work.
|
|
||||||
- A Guest is **sealed and singleton**: the Modules a Guest runs are fixed in the Guest file and are not overridable by a Host, and a Guest is instantiated at most once per Host, keyed by its namespace path.
|
|
||||||
Running more than one instance of a service on a Host is out of scope for this work.
|
|
||||||
|
|
||||||
### Placement interface (Host-side)
|
|
||||||
|
|
||||||
A Host instantiates a Guest by setting fields under `guests.<path>`.
|
|
||||||
The Guest owns its interior Modules; the Host owns only this placement.
|
|
||||||
|
|
||||||
| Field | Meaning | Default |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `enable` | Turn the Guest on for this Host | off |
|
|
||||||
| `backend` | Realization backend | `container` |
|
|
||||||
| `vlan` | Tagged VLAN the Guest lives on; maps to the Host's `br-vlan<id>` bridge by convention | required when networked |
|
|
||||||
| `mac` | The Guest's MAC address; set it to reuse an existing address | derived deterministically and surfaced when unset |
|
|
||||||
| `address` | Static address on the VLAN | unset, meaning DHCP |
|
|
||||||
| `mounts` | Map of guest path to host path, each with a per-mount `readOnly` | read-write per mount |
|
|
||||||
| `secrets` | Names of secrets the Guest needs | none |
|
|
||||||
| `limits` | `memory`, `cpu`, `tasksMax` caps applied to the guest's unit | uncapped |
|
|
||||||
| `nesting` | Grant the nested-container prerequisites so the interior can run Podman/OCI | off |
|
|
||||||
| `autoStart` | Start the Guest at boot | on |
|
|
||||||
|
|
||||||
- The `vlan` field maps to a bridge by the `br-vlan<id>` **naming convention**, so a Guest states only which VLAN it lives on.
|
|
||||||
- A **build-time assertion** ties a Guest's `vlan` to the set of VLANs its Host's network foundation declares, so a Guest on an undeclared VLAN fails the Host build with a clear message.
|
|
||||||
- The `mac` field is pinned inside the guest through the guest's own systemd-networkd, which is the only way a nested-container MAC is stable; an unset `mac` derives a stable address from the Guest's namespace path in a locally-administered range, readable via evaluation so the operator can add a reservation.
|
|
||||||
- The `mounts` field is realized as the nested container's bind mounts and defaults to read-write, matching the migration reality that services must write to pools.
|
|
||||||
- The `nesting` field is what makes the OCI fallback a plain Guest: the Skeleton emits the nested-container cgroup-delegation and capability prerequisites once, so the operator flips one boolean and the interior's `oci-containers` runtime works, with Podman as the default runtime.
|
|
||||||
|
|
||||||
### Bases and the shared environment
|
|
||||||
|
|
||||||
- The shared base config **splits in two** (recorded under the Skeleton term): a **host base** that carries host-only machinery (bootloader, hardware profile, host identity, boot and garbage-collection timers), and a slim **guest-base** that every nested Guest stands on.
|
|
||||||
- Every Guest **imports the full `modules/` tree**, so any Module is available to enable inside a Guest; only Modules whose needs the base meets are enabled in practice.
|
|
||||||
- A Guest gets **home-manager and the same primary user** as a Host, which makes every Module placement-agnostic and removes any need for a host-versus-guest Module taxonomy.
|
|
||||||
- The **guest-base auto-enables** the `toolkit` bundle and `modules.ssh`, so every Guest has the baseline toolset and SSH access without per-Guest wiring.
|
|
||||||
Reaching a Guest by `machinectl` from its Host needs no Guest configuration and is the always-available fallback.
|
|
||||||
|
|
||||||
### New and modified Modules
|
|
||||||
|
|
||||||
- **`modules.toolkit`** (new): a deliberate bundle Module turning on fish, tmux, nvim, git, and direnv as one unit.
|
|
||||||
The guest-base auto-enables it; a Host enables it explicitly, keeping the Host a full checklist.
|
|
||||||
This is a bundle wanted as a unit, distinct from the grouping-directory enables ADR 0004 rejected.
|
|
||||||
- **`modules.network`** (new): the host-level networking foundation.
|
|
||||||
A Host declares its trunk interface and the set of VLANs to materialize, and the Module emits one bridge per tagged VLAN with systemd-networkd and manages the Host's own management address.
|
|
||||||
- **`modules.zfs`** (new): the host-level pool import.
|
|
||||||
A Host declares its host id, the pools to import, and their dataset mountpoints; the pools are durable state that is imported, never rebuilt.
|
|
||||||
- **A shared `storage` group** with a fixed gid in the shared base gives 1:1 ownership between Host and Guest.
|
|
||||||
Because the container backend uses identity mapping, a guest service that writes as the `storage` group lands on the pool as that same group, which is the entire "no permission errors" mechanism.
|
|
||||||
- **`modules.ssh`** (reused): the guest-base enables it in a guest flavor, with the operator's authorized keys and a self-generated host key, since a Guest does not carry a per-Guest host identity the way a Host does.
|
|
||||||
- **The Skeleton** grows the Guest realization: the Auto-loader's discovery of `guests/`, the compilation of a Guest's interior into a nested container, and the wiring of placement fields (VLAN attachment, MAC pinning, bind mounts, secret mounts, unit caps, nesting prerequisites).
|
|
||||||
|
|
||||||
### Secrets
|
|
||||||
|
|
||||||
- The **Host is the sole decryptor**, consistent with the host identity of ADR 0002 and the existing sops-nix setup.
|
|
||||||
The Host decrypts, and a Guest receives only the specific secret files it names, read-only bind-mounted in, with ownership aligned by the identity mapping.
|
|
||||||
A Guest holds no age key.
|
|
||||||
|
|
||||||
## Testing Decisions
|
|
||||||
|
|
||||||
A good test here exercises **external, observable behavior of a Guest and its foundations**, not the internal shape of the generated nested-container config.
|
|
||||||
The load-bearing behaviors are: a Guest presents its own MAC and its own IP on the correct tagged VLAN, a guest service writes to a shared-group mount without permission error, the baseline toolset and SSH access are present, and a misplaced Guest fails the build.
|
|
||||||
|
|
||||||
Two seams, the fewest that cover the work:
|
|
||||||
|
|
||||||
1. **The Host toplevel build via `nix flake check`** — the existing, primary seam every Host already has.
|
|
||||||
This is where a Guest's evaluation, its placement fields, the base split, the Auto-loader wiring, and the `vlan`-against-declared-VLANs assertion are all verified.
|
|
||||||
Cheap targeted evaluations of derived values (a Guest's resolved bridge attachment, its derived MAC, the shared `storage` gid) ride on this same seam.
|
|
||||||
2. **One NixOS VM integration test**, added to the flake's `checks` so `nix flake check` runs it — a new seam, and the highest behavioral one available.
|
|
||||||
A single harness boots the `modules.network` foundation and one sample Guest and asserts the three hard requirements together: the guest has its own MAC, gets its own IP on a tagged VLAN across a virtual L2 segment, and can write to a bind-mounted directory owned by the shared `storage` group.
|
|
||||||
|
|
||||||
Prior art: the flake's `checks.<host>` toplevel builds are the established evaluation seam; the upstream NixOS test suite exercises nested-container networking, including macvlan and extra-veth cases, and is the model for the VM integration test.
|
|
||||||
The two behaviors a VM cannot honestly reproduce — real 802.1Q against the physical switch and real ZFS identity-mapped writes on the pool — are verified manually on the target Host rather than in the test.
|
|
||||||
|
|
||||||
## Out of Scope
|
|
||||||
|
|
||||||
- The `microvm` backend and any hard-isolation guest; the concept reserves the backend value but this work does not build it.
|
|
||||||
- Running more than one instance of a service on a Host; the Guest stays singleton and the operator will address multiplicity separately.
|
|
||||||
- Exotic OCI images that need their own nested init or unusual storage drivers, which are the future reason to reach for the microvm backend.
|
|
||||||
- The concrete homelab Host itself and its real values — its trunk interface name, VLAN ids, pool names, and per-Guest MAC and mount assignments — which are host-specific data supplied when a Host is added.
|
|
||||||
- Migrating specific services (the *arr stack, download clients, media servers) into Guests; this work delivers the concept and its foundations, not the service catalogue.
|
|
||||||
- Real-switch VLAN behavior and real ZFS identity-mapped writes, which are verified manually on the target Host.
|
|
||||||
|
|
||||||
## Further Notes
|
|
||||||
|
|
||||||
- The decision to introduce Guest as a third concept, backend-agnostic but container-only for now, sealed and singleton, is recorded in ADR 0006.
|
|
||||||
- The "no permission errors" result rests on the container backend's identity mapping being the privileged-container equivalent the operator already trusts from Proxmox; it is deliberately not the unprivileged idmap model, which is what made pool writes painful before.
|
|
||||||
- Pools are durable state imported by the Host, never recreated by a rebuild, so a service's data survives any rebuild or reimage.
|
|
||||||
- Podman is the default and recommended runtime inside a nesting Guest, matching what the operator already runs; a container-backend Guest is soft-isolated (shared kernel), the same isolation class as the Proxmox LXCs being replaced, so nothing is lost on that axis in the move.
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
A new bundle Module, `modules.toolkit`, that turns on the baseline interactive environment — fish, tmux, nvim, git, and direnv — as one unit.
|
|
||||||
Enabling it on a Host brings up all five together, so any Host or Guest shell feels identical.
|
|
||||||
This is a deliberate bundle wanted as a unit, distinct from the grouping-directory aggregate enables ADR 0004 rejected.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] `modules.toolkit` declares an `enable` option following the Enable convention and lives at a path its file location mirrors, per the Namespace convention.
|
|
||||||
- [x] Enabling `modules.toolkit` turns on fish, tmux, nvim, git, and direnv as a group.
|
|
||||||
- [x] A Host that enables `modules.toolkit` still reads as a flat checklist — the bundle is one line, not a hidden group of five.
|
|
||||||
- [x] A Host enabling `modules.toolkit` builds via `nix flake check`, and the five underlying Modules are enabled (verifiable by `nix eval` of their `enable` values).
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
- `modules/toolkit.nix` follows the sanctioned aggregator pattern of `modules/desktop/desktop.nix`: an index node whose `enable` sets each member's `enable = lib.mkDefault true`, so a Host can still override any single piece while the one flag brings up the bundle.
|
|
||||||
- `neogaia` was converted as the demonstrating Host: its five individual `modules.{fish,git,direnv,tmux,nvim}.enable = true` lines collapse to one `modules.toolkit.enable = true`.
|
|
||||||
- The bundle also sets `modules.fish.defaultShell`, so fish is the login shell wherever the toolkit is enabled — part of making any Host or Guest shell feel identical. This is `mkDefault`, so a Host can still opt out. `neogaia`'s previously explicit `defaultShell` line is therefore dropped.
|
|
||||||
- Verified: `nix flake check` builds `checks.x86_64-linux.neogaia`, and `nix eval` of each of the five members' `enable` on neogaia returns `true`.
|
|
||||||
- The working tree also carries `.claude/CONTEXT.md`, ADR 0006, and the `guests` spec — planning artifacts for the wider `guests` spec, not this task. They are deliberately left out of this task's commit and staged separately by the broader work.
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: 0001-toolkit-bundle
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The tracer bullet for the Guest concept: the thinnest complete path from discovery to a running nested container.
|
|
||||||
The Auto-loader gains a third kind, discovering every Guest under `guests/` the way it already discovers Modules and Hosts.
|
|
||||||
The shared base config splits into a host base (`system.nix`, carrying host-only machinery — bootloader, hardware profile, host identity, boot and garbage-collection timers) and a slim guest-base that every nested Guest stands on.
|
|
||||||
Both keep the primary user, home-manager, and the shared overlays; the guest-base additionally auto-enables the `toolkit` bundle and `modules.ssh`, and imports the full `modules/` tree so any Module is available inside a Guest.
|
|
||||||
A minimal sample Guest is Module-shaped: it declares its own `guests.<path>` namespace with an `enable` and a `backend` field (default `container`, `microvm` reserved but not built), and guards its body per the Enable convention.
|
|
||||||
Its body's payload — the one difference from a Module — realizes a nested container running the Guest's interior on the guest-base.
|
|
||||||
A Host enables the sample Guest exactly as it enables a Module, and the whole thing builds through the existing `nix flake check` seam.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] The Auto-loader discovers and wires every Guest under `guests/` as a third kind, with no manual `imports` edits, and a Guest's option path mirrors its `guests/` location per the Namespace convention (including the index-node and file-or-folder rules).
|
|
||||||
- [x] The shared base config is split into a host base and a slim guest-base; the existing Host still builds via `nix flake check` with its host-only machinery intact.
|
|
||||||
- [x] The guest-base includes the primary user, home-manager, and the shared overlays, auto-enables `toolkit` and `modules.ssh`, and imports the full `modules/` tree.
|
|
||||||
- [x] A sample Guest declares `guests.<path>.enable` plus a `backend` field defaulting to `container`, guards its body on `enable`, and realizes a nested container running its interior when a Host enables it.
|
|
||||||
- [x] The `microvm` backend value is accepted as reserved but unimplemented, failing clearly rather than silently building nothing.
|
|
||||||
- [x] A Host enabling the sample Guest builds via `nix flake check`, the guest is reachable from its Host by `machinectl` with no per-Guest configuration, and the baseline toolset and SSH access are present inside it.
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
- The base split is realized as three files, not two.
|
|
||||||
`base.nix` is the shared substrate both bases build on — the primary user, home-manager, the `unstable`/`stable` overlays, and flakes.
|
|
||||||
`system.nix` (the host base) imports it and adds the host-only machinery (bootloader limit, sops decryption and the password, the maintenance timers, the chaotic cache, console keymap).
|
|
||||||
`guest.nix` (the guest-base) imports it and adds the slim guest layer.
|
|
||||||
Factoring the common substrate out keeps "both include the primary user, home-manager, and the shared overlays" a single fact rather than a duplicated one.
|
|
||||||
|
|
||||||
- The guest-base imports `sops-nix` and `stylix` alongside the full `modules/` tree.
|
|
||||||
This is load-bearing, not incidental: the module system pushes an `mkIf` down to the leaves it guards, so an option path a module names must be *declared* even where its `enable` is off.
|
|
||||||
`modules/ssh.nix` names `sops.*` and the desktop modules name `stylix.*`, so those option namespaces have to exist for the tree to evaluate inside a guest that leaves them disabled.
|
|
||||||
|
|
||||||
- `modules/ssh.nix` gained a guest flavor.
|
|
||||||
`hostKeys.restore` (default on) gates restoring host keys from secrets, and both `hostKeys.sopsFile` and `userKey.sopsFile` are now nullable.
|
|
||||||
A guest sets `restore = false` and names no sops files, so its daemon self-generates a host key and it carries no age key — verified: the interior's `sops.secrets` is empty and `services.openssh.hostKeys` falls back to the generated defaults, while `neogaia` still restores its committed host keys with `openssh.hostKeys = [ ]`.
|
|
||||||
|
|
||||||
- The namespace mirroring is honored by author discipline, exactly as a Module's is: `my.guest { name = "sample"; }` names the option path, and `guests/sample.nix` places it.
|
|
||||||
The Auto-loader change is the same recursive `collectNixFiles`, so the index-node and file-or-folder rules a folder-shaped guest would use come for free from the loader that already serves Modules — no folder-shaped guest exists yet to exercise them.
|
|
||||||
|
|
||||||
- The guest gets `privateNetwork = true` by default, so its interior sshd never contends with the host's on the shared namespace.
|
|
||||||
It is `mkDefault`, so the networking foundation can later attach the guest to a VLAN bridge.
|
|
||||||
|
|
||||||
- `backend` is an `enum [ "container" "microvm" ]` defaulting to `container`.
|
|
||||||
`microvm` is built by nothing; choosing it trips a build-time assertion with a clear message rather than silently producing no container, per the acceptance criterion and ADR 0006's reserved-value decision.
|
|
||||||
|
|
||||||
- `neogaia` enables `guests.sample` to demonstrate the path end to end, the way it demonstrated `modules.toolkit`.
|
|
||||||
This means a rebuild starts a live nspawn container on the laptop; it is a minimal smoke test and can be turned off with one line.
|
|
||||||
|
|
||||||
- `nix flake check` passes and builds the nested `nixos-system-sample` interior in full.
|
|
||||||
A pre-existing nixvim warning about its own `nixpkgs.follows` now also prints for the guest's reused Neovim config; it is upstream noise, not a defect in this change.
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The host-level networking foundation, `modules.network`, that a Host declares once and every Guest attaches to.
|
|
||||||
A Host states its trunk interface and the set of VLANs to materialize, and the Module emits one bridge per tagged VLAN using systemd-networkd, named by the `br-vlan<id>` convention, and manages the Host's own management address.
|
|
||||||
This is a standalone host-level Module and does not yet wire any Guest to a bridge — that is the Guest networking placement slice.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] `modules.network` declares an `enable` option and its option path mirrors its file location per the Namespace convention.
|
|
||||||
- [x] A Host declares its trunk interface and its set of VLAN ids through the Module's options.
|
|
||||||
- [x] Enabling the Module emits exactly one systemd-networkd bridge per declared VLAN, each named `br-vlan<id>`, and manages the Host's own management address.
|
|
||||||
- [x] A Host enabling `modules.network` builds via `nix flake check`, and the emitted bridge names are verifiable by `nix eval`. Verified by temporarily enabling it on `neogaia`; the enablement is not committed (see notes).
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
Each VLAN materializes as three networkd entries: a `<trunk>.<id>` tagged sub-interface stacked on the trunk, a `br-vlan<id>` bridge, and a network enslaving the sub-interface to the bridge.
|
|
||||||
The trunk and every bridge set `RequiredForOnline = "no"`, so `systemd-networkd-wait-online` never blocks boot on a link with no carrier.
|
|
||||||
|
|
||||||
The management address takes a static CIDR, or DHCP when left null, on the management VLAN's bridge alone.
|
|
||||||
Two assertions guard it: the management VLAN must be one of the declared VLANs, and a declared management address must name a management VLAN, so an address can never be silently dropped for want of a bridge to carry it.
|
|
||||||
|
|
||||||
The Module owns its own NetworkManager `unmanaged` guard for the trunk, sub-interfaces, and bridges, so enabling it is self-sufficient on a host that also runs NetworkManager rather than pushing that wiring into every Host.
|
|
||||||
A `management.gateway` option was considered and dropped as speculative for this slice, since the foundation carries no other routing.
|
|
||||||
|
|
||||||
No host commits an enablement of this Module.
|
|
||||||
The repo's only host is `neogaia`, a wifi laptop on an access port, and enabling the Module there turns on `systemd-networkd` and pulls in `systemd-resolved`, which takes over the laptop's DNS.
|
|
||||||
That is an unwanted change to a daily machine that cannot present guests as L2 citizens anyway (wifi does not bridge), so the standing enablement waits for the first wired server host.
|
|
||||||
The build and bridge-name evaluation were verified by temporarily enabling the Module on `neogaia` (`nix flake check` passed, `nix eval` showed `br-vlan10`/`br-vlan20`), then reverting.
|
|
||||||
Both stay reproducible from the committed tree by enabling the Module ad hoc through `nixosConfigurations.neogaia.extendModules`, leaving the host file untouched.
|
|
||||||
|
|
||||||
No Guest is wired to a bridge — that is the guest networking placement slice.
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: [0002-guest-walking-skeleton, 0003-network-vlan-foundation]
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The Host-side placement fields that make a Guest a first-class L2 citizen on a tagged VLAN, exactly as Proxmox did.
|
|
||||||
A Host sets `vlan`, and the Guest attaches to that Host's `br-vlan<id>` bridge by the naming convention, so the Guest states only which VLAN it lives on.
|
|
||||||
A Host may set `mac` to reuse an existing address so its router's DHCP reservations keep working; an unset `mac` derives a stable, readable address from the Guest's namespace path in a locally-administered range, surfaced via evaluation so the operator can add a reservation.
|
|
||||||
The MAC is pinned inside the guest through the guest's own systemd-networkd, which is the only way a nested-container MAC stays stable.
|
|
||||||
A Host may set `address` for a static IP; unset means DHCP, keeping IP management centralized at the router.
|
|
||||||
A build-time assertion ties the Guest's `vlan` to the set of VLANs its Host's `modules.network` declares, so a Guest naming an undeclared VLAN fails the Host build with a clear message rather than as a broken bridge at runtime.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] A Host setting `guests.<path>.vlan` attaches the Guest to that Host's `br-vlan<id>` bridge by the naming convention.
|
|
||||||
- [x] Setting `mac` pins that exact address on the Guest via the guest's own systemd-networkd; leaving it unset derives a stable MAC from the Guest's namespace path in a locally-administered range, readable via `nix eval`.
|
|
||||||
- [x] Setting `address` gives the Guest a static IP on its VLAN; leaving it unset takes the address by DHCP.
|
|
||||||
- [x] A Guest whose `vlan` is not among its Host's declared VLANs fails `nix flake check` with a clear, actionable message naming the offending Guest and VLAN.
|
|
||||||
- [x] A Host with a correctly-placed networked Guest builds via `nix flake check`, and the Guest's resolved bridge attachment and derived MAC are verifiable by `nix eval`.
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
The `br-vlan<id>` naming was a local helper in `modules/network.nix` and is now a shared `bridgeName` in `lib.nix`, exported through `my` and consumed by both the network foundation and guest placement.
|
|
||||||
The convention has one source, so the bridge a guest attaches to can never drift from the bridge the host emits.
|
|
||||||
|
|
||||||
The interior networking is realized by a small module injected into the guest's container config only when `vlan` is set.
|
|
||||||
It enables the guest's own systemd-networkd on `eth0` — the name a nested container gives its bridged veth — pinning the placement MAC there and taking the static `address` or DHCP when it is unset.
|
|
||||||
Pinning the MAC through the guest's own networkd is the only way a nested-container MAC stays stable; the nspawn-assigned veth MAC is otherwise regenerated.
|
|
||||||
Enabling networkd default-enables `systemd-resolved`, so a DHCP guest also gets its resolver.
|
|
||||||
|
|
||||||
The derived MAC is the `mac` option's default, so an unset MAC reads back through `nix eval .#nixosConfigurations.<host>.config.guests.<path>.mac`.
|
|
||||||
The first octet is `02` (locally-administered, unicast) and the remaining five octets are a hash slice of the namespace path.
|
|
||||||
|
|
||||||
A static `address` sets only the on-VLAN IP, with no gateway or DNS.
|
|
||||||
This mirrors `modules.network`, which deliberately dropped a `management.gateway` as speculative for the foundation slice; off-VLAN routing for a statically-addressed guest is a later concern, and the centralized path stays DHCP.
|
|
||||||
|
|
||||||
The networked path is verified by `nix eval` against `neogaia` through `extendModules` rather than by committing an enablement, exactly as task 0003 verified `modules.network`.
|
|
||||||
`neogaia` is a wifi laptop that cannot bridge, and enabling networkd on it would take over its DNS, so its committed `guests.sample` placement leaves `vlan` unset.
|
|
||||||
Verified: with `vlan = 10` the guest resolves `hostBridge = br-vlan10` and the interior `eth0` networkd pins the derived MAC; an unset `address` yields `DHCP = "yes"` and a set one yields the static CIDR; and `vlan = 99` against declared `[10 20]` fails the build with the actionable message.
|
|
||||||
`nix flake check` passes with `guests.sample` building its interior in full.
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: 0002-guest-walking-skeleton
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The host-level pool-import foundation, `modules.zfs`, plus the shared `storage` group that makes identity-mapped pool writes work.
|
|
||||||
A Host declares its host id, the pools to import, and their dataset mountpoints; the pools are durable state that is imported, never rebuilt, so a service's data survives any rebuild or reimage.
|
|
||||||
A shared `storage` group with a fixed gid lives in the shared portion of the base config that both the host base and the guest-base include, giving 1:1 group ownership between a Host and its Guests.
|
|
||||||
This slice establishes the group and the pool import; a Guest actually writing to a mount as that group is the Guest storage placement slice.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] `modules.zfs` declares an `enable` option and its option path mirrors its file location per the Namespace convention.
|
|
||||||
- [x] A Host declares its host id, its pools, and their dataset mountpoints; enabling the Module imports those pools rather than recreating them.
|
|
||||||
- [x] A shared `storage` group with a fixed gid is defined in the shared base and is present identically on both a Host and its Guests.
|
|
||||||
- [x] A Host enabling `modules.zfs` builds, and the `storage` gid is verifiable by `nix eval`. Verified by ad-hoc enablement on `neogaia`; the enablement is not committed (see notes).
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
The pool import is realized by `boot.zfs.extraPools`, which imports the named pools rather than creating them, so a rebuild never touches pool contents.
|
|
||||||
`boot.supportedFilesystems = [ "zfs" ]` pulls the ZFS stack into the kernel and boot even on a host whose root is another filesystem, and `networking.hostId` is required because ZFS refuses to import a pool without a host id to stamp ownership onto.
|
|
||||||
Declared dataset mountpoints become plain `fileSystems` entries with `fsType = "zfs"`, orthogonal to the import: a pool with an empty map is still imported, leaving its datasets to their own ZFS `mountpoint` property.
|
|
||||||
|
|
||||||
`pools` is typed `attrsOf (attrsOf path)` — pool name to a dataset-relative-path to mountpoint map — rather than a per-pool submodule.
|
|
||||||
No per-pool option beyond the mount map is foreseen at host level, so the extra submodule layer would have been speculative.
|
|
||||||
|
|
||||||
The Module is `modules.zfs`, a flat single-file module, not `modules.storage.zfs` under a `storage/` directory.
|
|
||||||
Nothing else lives under a storage namespace, and the sibling host-level foundation `modules.network` is likewise flat, so the extra directory level would have grouped a single member.
|
|
||||||
The spec and ADR 0006 were updated to name it `modules.zfs` to match.
|
|
||||||
|
|
||||||
The `storage` group carries a fixed gid of 10000, placed in `base.nix` so a host and every guest built from this flake carry the identical number.
|
|
||||||
That identity is the whole write mechanism: an identity-mapped container write lands on the pool as the same numeric group with no per-service permission juggling.
|
|
||||||
10000 sits above the ids NixOS assigns automatically, so no generated account collides with it.
|
|
||||||
This slice only defines the group and the import; a Guest actually writing to a mount as this group is the Guest storage placement slice.
|
|
||||||
|
|
||||||
No host commits an enablement of this Module.
|
|
||||||
The repo's only host is `neogaia`, a laptop with a btrfs root, no ZFS pools, and a bleeding-edge CachyOS kernel whose `zfs-kernel-2.4.3` build is marked broken — so a committed ZFS enablement there would be both dishonest and unbuildable.
|
|
||||||
This mirrors the networking foundation's decision (task 0003) to verify by temporary enablement rather than commit one to a host that cannot honestly carry the feature.
|
|
||||||
The enabled build was verified through `nixosConfigurations.neogaia.extendModules`, enabling the Module against a declared pool and forcing `boot.kernelPackages = pkgs.linuxPackages` (a ZFS-supported kernel) so the incompatibility of *neogaia's* kernel choice does not mask the Module's own correctness; the full `nixos-system-neogaia` toplevel built.
|
|
||||||
The `storage` gid was verified identical (2000) on both the host config and the sample guest's interior, and `nix flake check` passes on the committed, module-inert tree.
|
|
||||||
The standing enablement waits for the first real storage host, supplied when that host is added.
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: [0002-guest-walking-skeleton, 0005-storage-zfs-foundation]
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The Host-side placement that lets a Guest mount shared pool paths at any granularity and write to them without permission errors.
|
|
||||||
A Host sets `mounts`, a map of guest path to host path, each with a per-mount `readOnly`, defaulting to read-write to match the migration reality that services must write to pools.
|
|
||||||
Each entry is realized as a bind mount of the nested container, so a Guest sees exactly the data it should — a single folder or a whole pool, read-only or read-write.
|
|
||||||
Because the container backend uses identity mapping, a guest service that writes as the shared `storage` group lands on the pool as that same group, which is the entire "no permission errors" mechanism and the privileged-container equivalent the operator already trusts from Proxmox.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] A Host setting `guests.<path>.mounts` bind-mounts each host path at its guest path inside the nested container.
|
|
||||||
- [x] Each mount honours its per-mount `readOnly`, defaulting to read-write.
|
|
||||||
- [x] Identity mapping is configured so a guest service writing as the `storage` group lands on the host pool as that same group with no permission error.
|
|
||||||
- [x] A Host with a Guest that has pool mounts builds, and the resolved bind mounts are verifiable by `nix eval`. Verified by ad-hoc mounts on the sample guest; no host commits mounts (see notes).
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
`mounts` is a placement option on the `guest` builder: an attribute set keyed by the guest-interior path, each value carrying `hostPath` and a `readOnly` flag that defaults to `false`.
|
|
||||||
It is realized as the nested container's `bindMounts`, where the container option's `mountPoint` defaults to the attribute key, so the guest path is stated once as the key.
|
|
||||||
The upstream `bindMounts` `isReadOnly` defaults to read-only, so it is driven explicitly from `readOnly` to make read-write the default here, matching the migration reality that services must write to their pools.
|
|
||||||
|
|
||||||
Identity mapping is pinned with `privateUsers = "no"`, which runs the container in the host's uid and gid space one to one.
|
|
||||||
That is the current NixOS default, but the option documents `"pick"` (a shifting map that would break pool writes) as its recommended value, so the property is pinned rather than left to a default that may drift.
|
|
||||||
With this mapping and the shared `storage` group from the storage foundation (identical gid on host and guest), a guest process writing as that group lands on a bind-mounted pool as the same group.
|
|
||||||
The write itself is verified manually on the target host per the spec's testing decisions, since a real identity-mapped ZFS write cannot be reproduced in the build.
|
|
||||||
|
|
||||||
No host commits pool mounts.
|
|
||||||
The repo's only host, `neogaia`, is a laptop with no ZFS pools, so a committed mount would point at a host path that does not exist and would fail the bind at container start.
|
|
||||||
This mirrors the storage and networking foundations (tasks 0005 and 0003), which verify by ad-hoc enablement rather than committing a placement a host cannot honestly carry.
|
|
||||||
The realization was verified through `nixosConfigurations.neogaia.extendModules`, setting two mounts on the sample guest: the resolved `containers.sample.bindMounts` carried the right `hostPath`, `mountPoint`, and per-mount `isReadOnly` (read-write and read-only), `privateUsers` resolved to `"no"`, the `storage` gid was identical on host and guest interior, and the full toplevel built.
|
|
||||||
`nix flake check` passes on the committed tree, where the sample guest declares no mounts.
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: 0002-guest-walking-skeleton
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The Host-side placement that gives a Guest only the decrypted secrets it names, while no Guest ever holds a decryption key.
|
|
||||||
A Host sets `secrets`, the names of the secret files the Guest needs.
|
|
||||||
The Host is the sole decryptor, consistent with the host identity and the existing sops-nix setup: it decrypts, and the Guest receives only the specific named secret files, read-only bind-mounted in, with ownership aligned by the identity mapping.
|
|
||||||
A Guest holds no age key.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] A Host setting `guests.<path>.secrets` gives the Guest exactly the named secret files and no others.
|
|
||||||
- [x] The named secrets are decrypted by the Host and bind-mounted into the Guest read-only, with ownership aligned by the identity mapping.
|
|
||||||
- [x] The Guest holds no age key and performs no decryption of its own.
|
|
||||||
- [x] A Host with a Guest that names secrets builds via `nix flake check`, and the resolved secret mounts are verifiable by `nix eval`.
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
`secrets` is a placement option on the `guest` builder: a list of secret names.
|
|
||||||
For each name the builder declares `sops.secrets.<name>` on the host, so the host is the sole decryptor from the sops files it already holds, and bind-mounts the decrypted file read-only into the guest at the same `/run/secrets/<name>` path it occupies on the host.
|
|
||||||
A service inside the guest therefore reads its credentials at the location it would on a host, keeping a module placement-agnostic.
|
|
||||||
|
|
||||||
The guest holds no age key and declares no secrets of its own.
|
|
||||||
This is inherited, not added: `guest.nix` stands on `base.nix`, not on the host base `system.nix` that sets `sops.age.keyFile`.
|
|
||||||
Verified through `nixosConfigurations.neogaia.extendModules`: with `guests.sample.secrets = [ "alexion-password" ]`, the interior's `sops.age.keyFile` is null and its `sops.secrets` is empty, while the host declares the secret and the container's `bindMounts` carries exactly the one entry, read-only, keyed and sourced at the secret's path.
|
|
||||||
|
|
||||||
Ownership alignment needs no new code.
|
|
||||||
The container already runs with `privateUsers = "no"` (task 0006), so the host and guest share one uid and gid space, and the decrypted file's host owner is its owner inside the guest.
|
|
||||||
A secret defaults to `root:root` mode `0400`, so a guest service running as a non-root user needs the operator to set `sops.secrets.<name>.owner` on the host, which merges cleanly with the builder's stub declaration.
|
|
||||||
The `secrets` field stays a list of names per the spec, which treats the owner as host-set data.
|
|
||||||
|
|
||||||
A build-time assertion rejects an in-guest path claimed by both a `mounts` entry and a secret, since the two attribute sets merge and the collision would otherwise resolve silently in the secret's favour.
|
|
||||||
|
|
||||||
No host commits a `secrets` placement, mirroring the mounts, networking, and storage foundations (tasks 0003–0006): the only host, `neogaia`, is a laptop carrying no service that names one.
|
|
||||||
`nix flake check` passes on the committed tree, where the sample guest declares no secrets.
|
|
||||||
The full host toplevel was built with the sample guest naming `alexion-password` (a real key in `secrets/shared.yaml`); sops-nix validates at build time that each named key exists in the host's sops files, so a build with a name absent from those files fails clearly rather than at activation.
|
|
||||||
Real identity-mapped reads on the target host are verified manually, per the spec's testing decisions.
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: 0002-guest-walking-skeleton
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The Host-side placement that caps a Guest's resources so one misbehaving service cannot starve its Host, plus the boot-start toggle.
|
|
||||||
A Host sets `limits` — `memory`, `cpu`, and `tasksMax` — applied to the guest's unit, uncapped by default.
|
|
||||||
A Host sets `autoStart` to control whether the Guest starts at boot, on by default.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] A Host setting `guests.<path>.limits.memory`, `.cpu`, or `.tasksMax` applies the corresponding cap to the guest's unit.
|
|
||||||
- [x] Each limit is uncapped when unset.
|
|
||||||
- [x] `autoStart` starts the Guest at boot by default, and disabling it leaves the Guest defined but not started at boot.
|
|
||||||
- [x] A Host with a capped Guest builds via `nix flake check`, and the resolved unit caps are verifiable by `nix eval`.
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
- The caps map to the guest's own systemd unit, `container@<name>.service`, which the container backend generates.
|
|
||||||
The guest module contributes `serviceConfig.MemoryMax`, `.CPUQuota`, and `.TasksMax`, and the module system merges these with the backend's own `serviceConfig` for that unit.
|
|
||||||
Only set caps appear: a `filterAttrs` drops any limit left null, so an unset limit contributes no key and systemd keeps its uncapped default rather than the module writing an explicit "infinity".
|
|
||||||
- `limits.memory` and `limits.cpu` are strings passed through to systemd verbatim (`2G`, `150%`), since systemd already parses size and percentage forms and re-inventing the parsing here would only narrow what the operator can express.
|
|
||||||
`limits.tasksMax` is a positive int, matching `TasksMax`'s count.
|
|
||||||
- `autoStart` is now a placement option defaulting true, and the container's `autoStart` reads from it directly rather than the previous `mkDefault true`.
|
|
||||||
The backend gates `wantedBy = [ "machines.target" ]` on `autoStart`, so `false` leaves the `container@<name>.service` unit fully defined but out of `machines.target` — startable on demand, not at boot — verified by evaluation.
|
|
||||||
- `neogaia`'s skeleton guest carries modest demonstrative caps (`memory = "1G"`, `cpu = "100%"`, `tasksMax = 512`), following task 0002's precedent of exercising the guest path on this host through its own `nix flake check`.
|
|
||||||
Its `autoStart` is left at the default so 0002's live boot smoke test is unaffected.
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: 0002-guest-walking-skeleton
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
The Host-side placement that makes the OCI fallback a plain Guest, so image-only software has a declarative home without a separate mechanism.
|
|
||||||
A Host sets `nesting` to grant the nested-container prerequisites so the Guest's interior can run Podman and other OCI containers.
|
|
||||||
The Skeleton emits the nested-container cgroup-delegation and capability prerequisites once, so the operator flips one boolean and the interior's `oci-containers` runtime works, with Podman as the default runtime.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [x] A Host setting `guests.<path>.nesting` grants the nested-container cgroup-delegation and capability prerequisites, off by default.
|
|
||||||
- [x] With `nesting` on, the Guest's interior can define `oci-containers` running Podman as the default runtime.
|
|
||||||
- [x] With `nesting` off, those prerequisites are absent and the Guest is unaffected.
|
|
||||||
- [x] A Host with a nesting Guest that defines an OCI container builds via `nix flake check`.
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
- `nesting` is a Host-side placement field on the guest, a bool defaulting off, alongside the other placement fields in `lib.nix`.
|
|
||||||
On, it grants the guest's container `CAP_NET_ADMIN` and the `/dev/net/tun` and `/dev/fuse` device nodes, the capability and devices an OCI runtime reaches for to network its containers and back their overlay storage.
|
|
||||||
Off, both `additionalCapabilities` and `allowedDevices` are empty, matching the NixOS defaults, so a non-nesting guest is untouched.
|
|
||||||
|
|
||||||
- The capability prerequisite is `CAP_NET_ADMIN` alone.
|
|
||||||
A container-backend guest runs privileged (`privateUsers = "no"`), so it already retains the broad nspawn capability set including `CAP_SYS_ADMIN`; the one addition an OCI runtime needs is network administration for its bridges and firewall rules.
|
|
||||||
|
|
||||||
- cgroup delegation is not toggled by `nesting`, a deliberate deviation from the criterion's wording that the flag "grants" it and that it is "absent" when off.
|
|
||||||
The NixOS container backend sets `Delegate = true` on every container's unit unconditionally, so the delegated cgroup subtree an OCI runtime manages is always present.
|
|
||||||
Re-emitting it under `nesting` would be redundant, and forcing it off for non-nesting guests to make it literally "absent" would remove a harmless, useful default for no gain.
|
|
||||||
The Skeleton records the prerequisite as satisfied-elsewhere with an absence pointer comment, so a reader does not think delegation was forgotten.
|
|
||||||
|
|
||||||
- A new `guests/nesting-sample.nix` carries an interior that defines an `oci-containers` workload, the payload the criteria exercise, and `neogaia` enables it with `nesting = true`.
|
|
||||||
This follows the walking-skeleton's precedent of proving a guest path through the one Host's `nix flake check`: the flake check builds the nested `nixos-system-nesting-sample` in full, pulling in `podman` and the generated `podman-hello.service` unit, which is how criteria two and four are verified on the build seam.
|
|
||||||
The sample carries the same modest caps as the walking-skeleton guest so an interior container cannot starve the laptop.
|
|
||||||
|
|
||||||
- The two behaviors the build seam cannot prove — that the interior Podman actually starts a container and that its networking works — are left to manual verification on the target Host and to the VM integration test of task 0010, per the spec's testing decisions.
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
---
|
|
||||||
spec: guests
|
|
||||||
blocked-by: [0004-guest-networking-placement, 0006-guest-storage-placement]
|
|
||||||
---
|
|
||||||
|
|
||||||
## What to build
|
|
||||||
|
|
||||||
One NixOS VM integration test, added to the flake's `checks` so `nix flake check` runs it, exercising the external observable behavior of a Guest and its foundations rather than the internal shape of the generated nested-container config.
|
|
||||||
A single harness boots the `modules.network` foundation and one sample Guest and asserts the three hard requirements together: the Guest presents its own MAC, gets its own IP on a tagged VLAN across a virtual L2 segment, and can write to a bind-mounted directory owned by the shared `storage` group.
|
|
||||||
The upstream NixOS test suite's nested-container networking cases (macvlan, extra-veth) are the model.
|
|
||||||
The two behaviors a VM cannot honestly reproduce — real 802.1Q against the physical switch and real ZFS identity-mapped writes on the pool — are out of this test and verified manually on the target Host.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
- [-] A NixOS VM test is added to the flake's `checks` and runs as part of `nix flake check`.
|
|
||||||
- [-] The test boots `modules.network` and one sample Guest on a virtual L2 segment.
|
|
||||||
- [-] The test asserts the Guest presents its own MAC distinct from the Host's.
|
|
||||||
- [-] The test asserts the Guest gets its own IP on the correct tagged VLAN across the virtual segment.
|
|
||||||
- [-] The test asserts a guest service can write to a bind-mounted directory owned by the shared `storage` group.
|
|
||||||
|
|
||||||
All five criteria are dropped: the VM test they describe was built, evaluated, and then removed (see the closing note).
|
|
||||||
|
|
||||||
## Implementation Notes
|
|
||||||
|
|
||||||
The test lives in `tests/guest-integration.nix` and is merged into `checks.x86_64-linux` as `guest-integration`, built through `pkgs.testers.runNixOSTest`.
|
|
||||||
The flake builds its nodes with the same `my`/`inputs` special arguments every configuration gets, passed through `node.specialArgs`, so the host node imports the real `modules/network.nix` and the real `my.guest` builder rather than a hand-rolled stand-in.
|
|
||||||
|
|
||||||
The harness is two nodes on one test-framework segment, the "virtual L2 segment".
|
|
||||||
The host runs `modules.network` with `trunk = "eth1"` and `vlans = [10]`, and one guest placed on VLAN 10.
|
|
||||||
A second `router` node speaks VLAN 10 only on a tagged `eth1.10` sub-interface and serves DHCP there, so the guest getting a `10.0.10.x` lease and the reverse `router → guest` ping succeed only when 802.1Q tagging works end to end across the segment.
|
|
||||||
This exercises the tagged path honestly rather than plain co-segment reachability.
|
|
||||||
The MAC assertion checks the guest's `eth0` equals its derived placement MAC and differs from the host trunk, and the storage assertion relies on the identity map (`privateUsers = "no"`) carrying gid 10000 through unshifted, so `stat -c %G` reading `storage` on the host is the "no permission errors" mechanism under test.
|
|
||||||
|
|
||||||
Booting a networked guest surfaced a latent defect in the guest networking foundation: enabling the guest's own networkd default-enables `systemd-resolved`, which conflicts with the nested-container default of inheriting the host's `resolv.conf`, and the guest's toplevel failed to build with "Using host resolv.conf is not supported with systemd-resolved".
|
|
||||||
Task 0004 never hit this because it only evaluated derived values, never built a networked guest's toplevel, and no committed host places a networked guest.
|
|
||||||
The fix is one line in `guestNet` (`networking.useHostResolvConf = false`), so any networked guest keeps its own resolver.
|
|
||||||
|
|
||||||
The guest interior here carries a storage-writing service as test scaffolding, since a real guest seals its own interior and the sample guest carries no service.
|
|
||||||
The host node also orders `container@sample` after the bridge's device unit, since the container enslaves its veth to the bridge at start and the upstream containers module orders only after `network.target`, not after the specific `hostBridge`.
|
|
||||||
|
|
||||||
Following the pattern of tasks 0003, 0004, and 0006, no committed host places a networked or pool-mounted guest, since the repo's only host is a wifi laptop with no bridge or pool.
|
|
||||||
The behaviors a VM cannot honestly reproduce — real 802.1Q against the physical switch and real ZFS identity-mapped writes on the pool — stay out of this test and are verified manually on the target Host, as the spec's testing decisions direct.
|
|
||||||
|
|
||||||
### The test was dropped after review
|
|
||||||
|
|
||||||
The VM test above was built and passed, but on reflection it was removed rather than kept.
|
|
||||||
Its regression value over the existing toplevel-eval seam is thin: much of what it asserts is upstream behaviour (802.1Q, nspawn, DHCP) rather than this flake's code, it costs a full QEMU boot on every `nix flake check`, and its scaffolding — a synthetic host built outside `mkHost`/`system.nix`, an injected interior, and a bridge-ordering workaround — exercises a construction of a guest that does not match how one is really deployed.
|
|
||||||
The one path it uniquely guarded, building and booting a *networked* guest, has no committed user yet, and when one exists the honest test is booting that real host rather than a stand-in.
|
|
||||||
|
|
||||||
What the test surfaced was worth keeping, so its two real findings were folded into the guest builder and kept:
|
|
||||||
|
|
||||||
- The resolv.conf fix in `guestNet` (`networking.useHostResolvConf = false`), so a networked guest keeps its own resolver.
|
|
||||||
- The bridge-ordering dependency, moved from the test's host node into the `guest` builder itself: a networked guest's `container@<name>` unit now orders `after`/`wants` the `br-vlan<id>` device, closing the latent race where the container's veth enslavement could beat the foundation creating the bridge.
|
|
||||||
|
|
||||||
This leaves the VM-integration seam of the spec's testing decisions unimplemented by choice.
|
|
||||||
Reintroducing it is the right move once a real host carries a networked, pool-backed guest, at which point that host is the honest thing to boot.
|
|
||||||
Reference in New Issue
Block a user