Status: plan of record for the native rewrite (2026-07-16). Visual checks on the dreamlake frontend are explicitly out of scope here (manual + design work); this plan covers everything automatable.

1. The shape of the problem

The control plane serves 113 routes; seven consumers sit on them with three different body encodings. The rewrite replaces the msgpack dispatch wire and the SDK surface — so the blast radius is exactly: who parses which bytes.

                 ┌────────────── JSON (field-name-sensitive) ──────────────┐
   TS CLI ───────┤                                                          │
   lakeshore-web ┤   resource CRUD routes (~90): providers/queues/storages/ │
   dreamlake-ai ─┤   workers/modes/secrets/... — UNCHANGED by the rewrite   │
    (via proxy)  └──────────────────────────────────────────────────────────┘
                 ┌────────────── msgpack (wire-rewrite blast zone) ─────────┐
   Python SDK ───┤   producer/submit|await, daemon/poll|ack|event,          │
   nymph (Rust) ─┤   payloads/presign + (new) result journal routes         │
                 └──────────────────────────────────────────────────────────┘

2. Consumer → surface matrix

Orphan routes (no consumer found; candidates to drop or intentionally reserve): GET /readyz, PATCH admin/namespaces/:slug, daemon key-rotation (daemon/keys, workers/:id/keys*), enroll-tokens* (server-internal), the namespaced workers/:id/exec/:eid/result alias, jobs/:id/result (both verbs), exec/:id/log[/stream].

3. What the rewrite changes vs preserves

Preserved (no consumer action): all JSON resource-CRUD routes; the dreamlake proxy model (until Phase 4); healthz; exec/log paths.

Changed: the msgpack dispatch wire (Envelope replaces the ad-hoc submit/poll/ack body shapes; unified wire.pack result path), SDK public surface (Queue/SyncQueue, no QueueFuture, string-key data returns), new journal routes (result-chunk append + cursor-read stream). During Phase 2/3 the Python daemon and CP move in lockstep; nymph poll/ack compatibility is a named gate — either the CP keeps the legacy poll/ack body shape for nymph's queues, or nymph's protocol.rs structs update in the same PR (decision at Phase 3 start, golden fixtures either way).

4. Test plan (automatable only)

A. Unit — SDK (runs on every commit, no network)

• wire.py: codec round-trips (ndarray/tensor/image/tuple/nesting), unknown-type raise, forward-compat ext codes, Envelope versioning, Frame reassembly across arbitrary chunk splits. ✅ landed (test_wire.py, 19 tests)
• context.py: local read/write/scope, shape-mirroring, key escaping, return-your-keys contract (all violation classes), local-simple invariant (fresh-interpreter import check: no httpx/msgpack on local paths). ✅ landed (test_context.py, 17 tests)
• Phase 2 additions: @udf dispatch matrix (sync/async/gen/asyncgen × local/remote), Queue/SyncQueue protocol core against a mock transport, idempotency-key semantics, check_returned_keys for async-gen accumulation.

B. Wire-contract conformance suite (the centerpiece)

One parametrized pytest suite, run against every dispatch-plane backend:

1. the minimal Python CP (in-process; every commit), and
2. the Node CP (full-stack harness via LAKESHORE_CP_DIR; pre-merge + CI).

Coverage — each item is a named test, not an aspiration:

• submit → claim → ack lifecycle; nack + retry; attempt increments
• atomic claim: N concurrent claimers, one job → exactly one winner
• fencing: a stale worker (job re-dispatched after unstale) cannot ack
• idempotent submit: same key twice → same invocation id, no dup job
• ordering: priority desc / FILO / Boltzmann-sampling distribution (χ² over many claims), all implemented as sorts/samples over the primary store
• counts == contents (zaku-desync regression): counts derived from the same rows that claim; assert after every lifecycle transition, incl. crash-mid-claim
• journal: N frames in-order, seq cursor reconnect resumes without loss/dup, explicit END vs dropped-connection distinguishable, ERROR reconstructs the exception, CHUNK runs reassemble, MAXLEN/TTL trim degrades to CP re-fetch
• streaming UDF: async-gen yields → one frame per yield → consumer iterates live; partial-key-list manifests concatenate; contract violation surfaces

C. Integration / e2e

• Existing self-provisioning full-stack e2e (Mongo + Node CP + Python daemon) updated to the new wire; runs pre-merge (E2E_REQUIRE_FULLSTACK=1 in CI).
• CP's own suite stays green — baseline 599 pass / 0 fail recorded 2026-07-16.
• nymph compat: golden msgpack fixtures for poll/ack request/response bodies, checked into both repos; a fixture diff = a deliberate, reviewed wire change.

D. Cross-consumer contract snapshots (catches field renames)

• Golden JSON snapshots for every response body the TS CLI, lakeshore-web, and dreamlake-ai parse (queues/workers/storages/invocations/providers lists + details). Generated against the Node CP, replayed against the minimal CP.
• dreamlake-server proxy suite stays green — baseline 108/109 (1 failure pre-existing on main: pipelines.test.ts PATCH-artifacts, documented 2026-07-16, unrelated to lakeshore).

E. Consumer pipelines (this repo = the real customer)

• Local golden run: stub-UDF generate_scene + annotate_subtasks under dls.scope, asserting the exact source/ · process/ · outputs/{i:04d}/ tree and manifest keys. Runs with zero network (the local fast path IS the test).
• One remote smoke per release: splats_to_mesh-shaped UDF through the minimal CP end-to-end (submit → daemon → storage side channel → keys back).

F. Examples as tests

• lakeshore-examples/test_runner.py: the no-CP examples (02, 05) run in CI as plain scripts; queue/gather examples (08, 09, 10) run against the minimal CP. Examples are migrated with the SDK in Phase 2 and become its acceptance tests.

G. Explicitly out of scope (manual / needs design)

• Visual checks on the dreamlake frontend (lakeshore panels, provider UI).
• TUI rendering (lakeshore tui).
• Cloud-provider launch paths (EC2/GCE/kube) — covered by existing CP tests only.

5. Gates per phase