DreamLake
  1. Get Started
  2. CLI
  3. Python
  4. Nymph
  5. API
  6. Admin

10 · Code mount

The "your local imports just work on the worker" story. On submit, the SDK archives the current git tree, uploads it under a content-addressed key, and stamps the archive onto the envelope. A worker that cannot import your module fetches and extracts the archive, then retries the import.

Exercises

  • HttpDispatch.push_code() — called automatically by UDF.submit for reference-transport functions when the plane has a code channel.
  • code_archive.push_if_needed — git archive HEAD → presigned PUT on the code-staging storage → register on POST /v1/namespaces/:ns/code-repos.
  • lakeshore code push / lakeshore code list — the same channel, by hand.
  • Worker-side: fetch by archiveKey, extract under $LAKESHORE_HOME/code/<commit>, prepend to sys.path, re-import.
  • Dedup: a (remote, commit) already registered on the control plane is never re-uploaded, and the worker reuses an already-extracted commit.

Requires

  • Everything in 05 · Hello, queue-bound UDF, against a control plane (--url / LAKESHORE_URL). The local SQLite plane has no code channel — it assumes a shared filesystem.
  • Everything in 09 · Storage round-trip: a storage entry named code-staging for the archives.
  • A git repository with a clean working tree. git archive HEAD ships committed content only; push_if_needed returns None on a dirty tree unless it is called with allow_dirty=True (lakeshore code push --dirty).

Verifies

A @udf whose module is not installed on the worker still runs there: the worker's ImportError triggers a fetch of the submitted archive and the import succeeds on the retry.

Run

Push the archive explicitly and confirm it registered:

bash
lakeshore code push
lakeshore code list

Then submit against the control plane, from inside the repo:

bash
export LAKESHORE_URL=http://localhost:8080
export LAKESHORE_CLIENT_TOKEN=<token>
python my_project/submit.py

with a worker running on the same plane:

bash
lakeshore worker start --queue default --url http://localhost:8080 --token <token>
The bundled example predates this path

lakeshore-examples/_legacy/11-code-mount exercises the archive channel through lakeshore exec, not through @udf. It is still a valid check of the upload half; the worker-side import path is exercised by the SDK submit above.

Expected output

The UDF's return value comes back, and the worker imported a module it did not have on disk when it started. The first submit on a new commit uploads (slow); a second submit on the same commit skips the upload entirely — that is the dedup signal.

If it fails

SymptomLikely cause
WorkerError: cannot import module ...No archive was stamped on the envelope. Dirty working tree, not a git repo, or the plane is the local SQLite one (no code channel). Commit, then resubmit.
Your edits do not take effect on the workergit archive HEAD ships committed content only — uncommitted changes never travel.
WorkerError: ... source differs from the submitted versionThe worker resolved a different revision of the function. Update its checkout, or pass --runtime-policy off for same-machine dev.
Upload fails or hangsThe code-staging storage entry is missing or its credentials are wrong. lakeshore code push shows the raw failure.
Slow on every run (no dedup)The commit changed every run (amend/rebase loop), or the worker's LAKESHORE_HOME/code cache is not persistent.

Status

Manual.

Next

→ 11 · Storage access from a UDF