08 · Bootstrap a daemon on a remote host

The "turn a vanilla box into a worker" flow. One command mints a token, SSHes in, drops the nymph binary and its config, starts it, and waits for the Worker row to appear.

Exercises

lakeshore daemon install <ssh-alias> — the four-step bootstrap.
Per-daemon token minting through POST /v1/namespaces/:ns/tokens.
Target-triple detection on the remote host.
Binary download from Cloudflare R2 (latest/, no checksum step — the .sha256 sidecars are used by scripts/install.sh and the OTA path, not by this bootstrap).
Outbound-only registration via POST /v1/daemon/hello.

Requires

Everything in 07 · SSH provider smoke.
An entry in ~/.ssh/config — daemon install takes an SSH alias, not a provider name, and hands it straight to ssh.
LAKESHORE_ADMIN_TOKEN exported (or --admin-token), because the mint route is admin-gated.
A writable home directory on the remote plus curl on PATH.
Outbound HTTPS from the host to the control plane and to R2.

Verifies

A fresh box becomes a registered Lakeshore worker through one command, with no inbound port opened on the host.

Run

bash

export LAKESHORE_URL=http://localhost:8080
export LAKESHORE_ADMIN_TOKEN=…

lakeshore daemon install my-box --tag gpu --runner process
lakeshore daemon list

Useful flags:

Flag	Effect
`--label <name>`	Worker label prefix. Defaults to the alias; a launch ULID is appended.
`--tag <name>`	Repeatable capability tag.
`--runner <name>`	Repeatable; defaults to `process`.
`--keep-alive-s <n>`	`-1` pool mode (the default here), `0` single job, `N` seconds idle.
`--binary <path>`	rsync a local nymph build instead of curling R2. Mutually exclusive with `--nymph-base-url`.
`--slurm` / `--slurm-partition <p>`	Wrap the launch in an `sbatch` heredoc instead of `setsid nohup`.
`--no-wait`	Skip the registration poll.

What it does on the host

~/.local/share/nymph/launches/<launchId>/
  nymph             executable
  udf-daemon.toml   config
  token             minted per-daemon token
~/.local/share/nymph-<launchId>.log   stdout + stderr

Everything lives under $HOME and runs as whoever you SSH'd in as — the script never sudos into another uid. It does make one best-effort sudo -n usermod -aG docker attempt so the daemon can reach the Docker socket, and ignores the failure when there is no sudo or no docker group.

The daemon is started detached with setsid nohup … in a background subshell (or an sbatch heredoc under --slurm). Target-triple detection maps uname -s to unknown-linux-gnu / apple-darwin and uname -m to x86_64 / aarch64, then curls <base>/dreamlake/nymph/latest/nymph-<triple>. Any other OS or arch exits with an error.

Expected output

→ [1/4] minting per-daemon token from http://localhost:8080…
  ✓ token minted (name 'daemon-my-box-<launchId>')
  launch ID: <launchId>
  log: ~/.local/share/nymph-<launchId>.log
→ [2/4] ssh my-box — bootstrapping (binary + config + start)…
  ✓ [3/4] daemon process started on my-box (label 'my-box-<launchId>')
→ [4/4] waiting (up to 60s) for registration ···
  ✓ daemon registered (worker 6712a…, label 'my-box-<launchId>')

The registration poll caps at 60 seconds and then fails; the daemon may still come up afterwards, so check daemon list before re-running.

If it fails

Symptom	Likely cause
`daemon install needs LAKESHORE_ADMIN_TOKEN`	Exit 2 before anything happens. Export it or pass `--admin-token`.
`ssh my-box exited with code …`	The alias does not resolve, or the remote shell rejected the heredoc. Try `ssh my-box true` on its own first.
`curl: command not found` on the host	Install `curl`, or use `--binary` to rsync a local build instead.
`unsupported OS` / `unsupported arch`	Only linux and macOS on x86_64 / aarch64 have published binaries.
Process started but never registers	Outbound HTTPS blocked, or the control-plane URL baked into the config is unreachable from the host. SSH in and read `~/.local/share/nymph-<launchId>.log`.
Registers, then vanishes minutes later	`keep_alive_s`. `-1` is pool mode; `0` exits the moment it goes idle; `N` exits after N idle seconds.

Status

Manual.

→ 09 · Storage round-trip