Living dev note. Iterate freely.

Two command classes, fundamentally different:

The current exec channel handles class 1. Sessions need a new channel — request/response can't model an open pipe.

Where does session state live?

A session IS a pipe. Pipes are in-kernel; you can't move them. So the bash process and its pipes must live on the daemon — there's no other option. The real design question is what metadata the controlplane carries to make sessions usable across CLI invocations.

Daemon owns (always)

If the daemon restarts, the session is dead. That's fine: every subprocess the user spawned was already dead too. Don't fake persistence we can't deliver.

Controlplane owns (for discoverability + reconnect)

Output bytes live in two tiers (see Log storage):

• Tier 1 — Redis Streams while the session is active. Hot, sub-ms, bounded by MAXLEN. Three streams per session: session:<id>:stdout, :stdin, :control. Bidirectional (stdin from CLI, stdout from daemon, signals like SIGWINCH).
• Tier 2 — S3 when the session ends. Controlplane flushes the full output buffer to a single archive key. Lifecycle policies handle retention.

Reconnect: CLI fetches the latest stream cursor from Redis; missed bytes replay automatically via XREAD from that id.

The simplest cut (v0)

CLI                         Controlplane                       Daemon
─────────────────────────────────────────────────────────────────────
POST /sessions
  worker_id, …          →   create record         →   spawn bash, return socket
                            session_id ←
SSE/WS /sessions/:id/stream  proxy bytes  ⇄   tail bash.stdout, feed bash.stdin
                                              │
                            on disconnect:    │   keep bash alive
                            mark `detached`   │   (grace 5 min)
                                              │
SSE/WS /sessions/:id/stream  reconnect  ⇄    replay last 4KB, resume tailing
                            mark `active`     │
                                              │
DELETE /sessions/:id   →    terminate   →    SIGTERM bash, drop record

Daemon-side:

struct Sessions {
    inner: HashMap<SessionId, Session>,
}

struct Session {
    child: tokio::process::Child,   // bash
    stdin: ChildStdin,
    output_tee: broadcast::Sender<Bytes>,  // tail subscribers
    redraw_buf: Mutex<VecDeque<u8>>,       // last 4KB
    last_activity: AtomicI64,
    state: SessionState,                    // active | detached
}

Controlplane-side: just the Session model with the metadata fields above. Zero output bytes stored.

CLI-side: lakeshore session <daemon> → drops you into a streaming client that proxies the local terminal to the SSE/WS. lakeshore session list shows all sessions; lakeshore session attach <id> reconnects.

Why this stays simple

• One source of truth per concern. Bytes → daemon. Discovery → controlplane.
• No mongo blob storage for output. The 16MB doc cap + write rate would dominate. Daemons are local to bytes; let bytes live there.
• Reconnect is "replay last 4KB then resume tailing". Not "scroll back through 30 minutes of training output". If users want scroll- back, that's an explicit tee to a file on the daemon — separate feature.
• Daemon restart kills sessions by design. Don't pretend otherwise; users should know their bash died.

Why NOT pure daemon (no controlplane record)

• lakeshore session list would have to poll every daemon → bad UX
• No central auth path for "who can attach to this session"
• Reconnect requires knowing the daemon — fine if you do, awful if you don't
• Multi-CLI attach is hard to coordinate without a shared state authority

Why NOT full controlplane persistence (output in mongo)

• 16MB document cap on the Worker row, or hundreds of writes/sec to a separate Output collection per session — pick your poison
• Output is high-throughput, low-value-after-a-few-seconds. Storing it amplifies cost for ~zero benefit.
• If users want durable output, the right tool is tee out.log and shipping that log out-of-band (s3 cp etc.). Don't put the daemon in the persistence business.

Open questions

• Should detached sessions auto-terminate after N minutes? Yes — start with 5 min grace, configurable. Otherwise zombies pile up.
• TTY allocation by default? Yes for interactive sessions. The one-time exec stays pipe-only.
• Auth shape for the SSE/WS endpoint? Same bearer as the rest of the namespaced routes. Session_id is not secret — auth proves membership.
• Resize signals (SIGWINCH)? Need a control channel on the same socket — small JSON header frames interleaved with byte frames. Or a separate POST /sessions/:id/resize endpoint (simpler, +1 round-trip per resize, fine).

Implementation order

1. Daemon-side session manager — Sessions struct + lakeshore daemon session-spawn debug verb that connects locally via Unix socket. Proves the bash + PTY + I/O machinery without any controlplane.
2. Controlplane Session model + create/list/terminate endpoints. No stream proxy yet.
3. Stream proxy: SSE first (simpler), WS later if SSE limits hit. Needs Heroku WS sanity check (per Deploy options H12 cap).
4. CLI: lakeshore session <daemon> + session list / attach.
5. Detach + reconnect — only after 1–4 work end-to-end.

Sequence-wise, this depends on the streaming-exec wire from Log storage (Tier 1 — Redis Streams). Sessions are essentially "streaming exec with a persistent process on the other end". Land streaming exec first; this follows naturally.