Kits batteries

A kit is data: it has roles, each role has typed settings, members export values to each other, and kits declare the secrets they generate. An assembly is one use of a kit: its kit is a ref, and each role takes the machines where its at predicate holds, the same predicates an aspect uses. See Assemblies and roles for the model by hand, and the glossary for the words. You keep the words: here the machine kind is machine and the member kind is member.

This example has a borg backup kit with a server and clients, one assembly of it, and the rules that connect them.

# Kits and assemblies: a kit is data; an assembly places machines in its roles with `at` predicates.
{ yen, lib, ... }:
let
  kits = yen.lib.batteries.kits;
  inherit (yen.lib) opt holds same;
  attrs = yen.lib.attrs;
  assemblies = lib.attrValues yen.assembly; # spawns read data
  # tree requests: every assembly and machine, with their data
  allAssemblies.instances = yen.kinds.assembly;
  allMachines.instances = yen.kinds.machine;
in
{
  yen.kinds.fleet = {
    spawns.machine = _: yen.machine;
    sinks.out.reduce = yen.lib.reduce.merge;
  };
  yen.kinds.machine = {
    fields.storage = opt yen.lib.bool false;
    # one member scope per (assembly, role) the machine takes
    spawns.member = kits.members { inherit assemblies holds; }; # (1)
    sinks.config = {
      reduce = yen.lib.reduce.merge;
      out =
        { scope, sinks, ... }:
        {
          config.${scope.name} = sinks.config;
        };
    };
  };
  yen.kinds.member = { };
  yen.kinds.kit.fields = {
    # (2)
    roles = opt attrs { };
    constraints = opt attrs { };
    exports = opt (yen.lib.listOf yen.lib.str) [ ];
  };
  yen.kinds.assembly.fields = {
    # (3)
    kit = yen.kinds.kit;
    roles = opt attrs { };
  };
  yen.kinds.fleet.sinks.exports = {
    reduce = kits.exports { isMember = holds yen.kinds.member; }; # (4)
    out =
      { sinks, ... }:
      {
        exports = sinks.exports;
      };
  };
  yen.fleet.main = { };
  yen.machine = {
    nas.storage = true;
    web = { };
  };
  yen.flake = { out, ... }: out;

  # what a kit is: its roles, a typed interface per role, limits, the exports it may give
  yen.kit.borg = {
    # (5)
    roles.server = { };
    roles.client.interface.dir = {
      type = yen.lib.str;
      default = "/var/backup";
    };
    constraints.roles.server.max = 1;
    exports = [ "endpoint" ];
  };
  yen.assembly.daily = {
    # (6)
    kit = yen.kit.borg;
    roles.server.at = m: m.storage;
    roles.client = {
      at = yen.kinds.machine;
      settings = m: lib.optionalAttrs (same m yen.machine.web) { dir = "/srv/backup"; };
    };
  };

  yen.aspects = {
    contract = {
      at = yen.kinds.fleet;
      cycle =
        { tree, ... }:
        {
          tree = [
            allAssemblies
            allMachines
          ];
          errors = [
            (kits.contract {
              # (7)
              assemblies = tree.select allAssemblies;
              machines = tree.select allMachines;
              inherit holds;
            })
          ];
        };
    };
    settings-typed = {
      at = yen.kinds.member;
      errors = kits.settings-typed; # (8)
    };
    # once per member. The server exports its endpoint; clients read it back.
    borg-server = {
      at = yen.lib.all [
        yen.kinds.member
        (s: same s.kit yen.kit.borg && s.role == "server")
      ];
      exports = # (9)
        { scope, ... }:
        {
          endpoint = "${scope.parent.name}:22";
        };
    };
    borg-client = {
      at = yen.lib.all [
        yen.kinds.member
        (s: same s.kit yen.kit.borg && s.role == "client")
      ];
      config = # (10)
        { scope, exports, ... }:
        {
          backup.${scope.assembly.name} = {
            inherit (scope.settings) dir;
            to = map (x: x.value.endpoint) (
              lib.filter (x: same x.assembly scope.assembly && x.role == "server") exports
            );
          };
        };
    };
    # one assembly gets more: an aspect at its members
    daily-verbose = {
      at = yen.lib.all [
        yen.kinds.member
        (s: same s.assembly yen.assembly.daily && s.role == "client")
      ];
      config.verbose = true;
    };
  };
}

(1) gives each machine one member scope for each (assembly, role) it takes. Settings are layered: the interface default, then the role’s.
(2) and (3) are the fields of a kit and of an assembly. The kit of an assembly is a ref.
(4) is the reduce of the exports sink. Members write their own element, and an export that the kit does not list is an error.
(5) is the kit. It has two roles, a typed interface for the client, a limit of one server, and the export endpoint.
(6) is the assembly. The server is any machine with storage. The settings of the client role are a function of the machine.
(7) checks the kits, assemblies and machines. It reports a kit that is no kit, an unknown role, an at that is no predicate, and a role with too many or too few machines.
(8) checks that every setting of a member is set and has its type.
At (9), the server exports its endpoint. At (10), each client reads the exports of its own assembly.

A kit adds no name outside itself: its data are values of the kit (yen.kit.disko.layouts), its aspects live in the kit (yen.kit.sshd.aspects.server), and the assemblies it adds are parts of the assembly that adds them. Each client reads the server’s endpoint from the exports of its assembly, with its own settings:

{
  nas = {
    backup = {
      daily = {
        dir = "/var/backup";
        to = [
          "nas:22"
        ];
      };
    };
    sshPub = "absent";
    verbose = true;
  };
  web = {
    backup = {
      daily = {
        dir = "/srv/backup";
        to = [
          "nas:22"
        ];
      };
    };
    sshPub = "ssh-ed25519 AAAAweb";
    verbose = true;
  };
}

Vars

A kit also declares the secrets it generates. A generator says which files a secret has, and which are secret. The plan says, for each machine, where each file goes and who can read it.

# vars: generators declared by kits, a plan per machine, public values read at eval time.
{ yen, lib, ... }:
let
  vars = yen.lib.batteries.vars;
  machineOf = s: if yen.lib.holds yen.kinds.member s then s.parent else s;
in
{
  yen.kinds.fleet.sinks = {
    generators = {
      reduce = vars.generators { inherit machineOf; }; # (1)
      out = vars.vars-plan {
        # (2)
        from = yen.kinds.fleet.sinks.generators;
        machines = lib.attrNames yen.machine;
        recipientsOf = m: [ "age1admin" ];
      };
    };
    public-reads.reduce = yen.lib.reduce.elements;
  };
  yen.aspects.ssh-vars = {
    at = yen.kinds.machine;
    # a function of the scope: one per machine (static content goes once per receiver)
    generators =
      { scope, ... }:
      {
        name = "ssh";
        files.key.secret = true;
        files."key.pub".secret = false;
      };
    config =
      { scope, ... }:
      {
        sshPub = vars.public-value { root = ../vars/per-machine; } scope.name "ssh" "key.pub" "absent"; # (3)
      };
  };
  yen.aspects.borg-vars = {
    at = yen.lib.all [
      yen.kinds.member
      (s: yen.lib.same s.kit yen.kit.borg && s.role == "client")
    ];
    generators =
      { scope, ... }:
      {
        name = "borg-${scope.assembly.name}";
        dependencies = [ "ssh" ];
        files.passphrase.neededFor = "users";
      };
  };
  yen.aspects.secret-reads = {
    at = yen.kinds.fleet;
    errors = vars.secret-reads {
      # (4)
      from = yen.kinds.fleet.sinks.generators;
      reads = yen.kinds.fleet.sinks.public-reads;
    };
  };
}

(1) reduces the generators of every machine: a shared generator once, the rest for each machine. A missing dependency is an error. (2) is the plan for each machine: secret files with their paths, recipients and generation order. (3) reads a public var at eval time, with a default. (4) reports an eval-time read of a secret file. This is the plan of web:

[
  {
    file = "passphrase";
    generator = "borg-daily";
    neededForUsers = true;
    path = "/run/secrets-for-users/vars/borg-daily/passphrase";
  }
  {
    file = "key";
    generator = "ssh";
    neededForUsers = false;
    path = "/run/secrets/vars/ssh/key";
  }
]

The contract

borg allows one server. A second machine with storage makes two:

[
  "kits: assembly daily role server: 2 machines, at most 1"
]

A name is not a ref. A string where the kit goes, and a string where a role’s machines go, are errors:

# a name where a ref goes: the kit, and a role's machines
{ yen, ... }:
{
  yen.assembly = {
    weekly = {
      kit = "borg";
      roles.server.at = yen.machine.nas;
    };
    hourly = {
      kit = yen.kit.borg;
      roles.server.at = "nas";
    };
  };
}

[
  "kit wants a kit ref"
  "kits: assembly hourly role server: `at` is a string, not a predicate"
  "kits: assembly weekly: kit is a null, not a kit"
]

The batteries

The batteries of yen.lib.batteries.kits:

Battery	Value of	Does
`members { assemblies; holds; }`	`spawns.<member kind>` of a machine kind	one member per (assembly, role), with refs to its assembly and kit; settings layered: interface default, then the role’s (attrs, or a function of the machine)
`exports { isMember; }`	`reduce` of the exports sink	members write their own element; another scope overrides an entry by refs; an unlisted export is an error
`contract { assemblies; machines; holds; }`	`errors`	a kit that is no kit, an unknown role, an `at` that is no predicate, no roles, min and max machines per role, `maxAssemblies`, two assemblies with one name
`assemblies own`	a list of assemblies	the assemblies and every part their kits add, at every depth: each part has a `path` and a `name` (the path joined by `-`)
`settings-typed`	`errors` at a member	every interface setting is set and has its type

The batteries of yen.lib.batteries.vars:

Battery	Value of	Does
`generators { machineOf; }`	`reduce` of the generators sink	shared generators once, the rest per machine; a missing dependency is an error
`vars-plan { from; machines; recipientsOf; }`	`out` of the generators sink	per machine: secret files with their paths, recipients, generation order
`secret-reads { from; reads; }`	`errors`	an eval-time read of a secret file
`public-value { root; }`	a function	a public var read at eval time, with a default

Working kits that use these batteries: Kits for services, Bundles and the templates kits-base and kits-homelab.

Every block on this page is a file of the template batteries-kits. Copy it and run it:

nix flake init -t github:vic/yen#batteries-kits

flake.nix
Directorymodules/
- inventory.nix
- vars.nix
Directoryvariants/
- by-name.nix
- two-servers.nix
Directoryvars/
- Directoryper-machine/
  - Directoryweb/
    Directoryssh/
    Directorykey.pub/
    value

Contribute Community Sponsor