# Storage, code, and mounts Part of the [CLI examples cookbook](/cli/examples.md). Auth setup lives on [Auth + setup](/cli/examples/auth.md). ## Storage `lakeshore storage` registers S3-compatible object stores. Note the verb is **singular** — `storage`, not `storages` — even though every sibling group is plural. Two kinds: | Kind | Meaning | Required config | | ----------- | -------------------------- | -------------------- | | `s3` | A full bucket | `bucket` | | `s3-prefix` | A scoped directory source | `bucket`, `prefix` | Both also accept `region`, `endpoint` (for MinIO / R2 / Ceph / Backblaze), and `creds.$secret` pointing at a registered `aws_keypair` secret. ### `storage add` — register a backend The common fields have direct flags. `--creds ` is shorthand for `--kwarg creds.$secret=`. ```bash lakeshore storage add my-s3 \ --kind s3 \ --bucket my-bucket \ --region us-east-1 \ --creds aws-prod-keys ``` ```bash lakeshore storage add training-data \ --kind s3-prefix \ --bucket my-bucket \ --prefix datasets/imagenet \ --creds aws-prod-keys \ --description "read-only imagenet source" ``` Non-AWS S3 needs `--endpoint`: ```bash lakeshore storage add r2-cache \ --kind s3 \ --bucket cache \ --endpoint https://.r2.cloudflarestorage.com \ --creds r2-keys ``` Longer configs can come from a file, and `--kwarg` sets arbitrary config fields with dotted keys nesting: ```bash lakeshore storage add my-s3 --kind s3 --config-file /tmp/storage.yaml lakeshore storage add my-s3 --kind s3 --kwarg bucket=my-bucket --kwarg 'creds.$secret=aws-prod-keys' ``` > **Note:** `storage add` calls Commander's `allowUnknownOption()`. Any other > `-- ` pair you pass is folded into the config object as > `key=value`. That means a typo'd flag silently becomes a config field > instead of erroring — check `storage show` after adding. ### `storage add --provision` — create the bucket too Requires `creds` to resolve to an `aws_keypair` secret. The controlplane calls `CreateBucket` on your behalf; it's idempotent if the bucket already exists under the same account. `--s3-option` is repeatable and is forwarded verbatim as a `CreateBucket` parameter. ```bash lakeshore storage add new-bucket \ --kind s3 \ --bucket lakeshore-scratch-42 \ --region us-east-1 \ --creds aws-prod-keys \ --provision \ --s3-option ACL=private ``` ### `storage list` / `show` ```bash lakeshore storage list lakeshore storage list --json lakeshore storage show my-s3 lakeshore storage show my-s3 --json ``` ### `storage update` — patch the config `--kwarg` deep-merges (pass `null` to drop a key); `--config-file` replaces the config wholesale. `--description` updates the description. ```bash lakeshore storage update my-s3 --kwarg region=us-west-2 lakeshore storage update my-s3 --description "prod artifacts" ``` ### `storage remove` — delete the entry Deletes the record only. `--purge` also deletes the S3 bucket, which needs credentials in the config. ```bash lakeshore storage remove my-s3 lakeshore storage remove my-s3 --purge ``` ### `storage presign` — a presigned URL Two positionals: the storage name and the object key (relative to the storage prefix). Defaults to a GET; `--put` presigns an upload. `--expires-in` defaults to 3600 seconds and is capped at 86400. ```bash lakeshore storage presign my-s3 path/to/artifact.tar.gz lakeshore storage presign my-s3 path/to/artifact.tar.gz --expires-in 600 lakeshore storage presign my-s3 uploads/new.bin --put ``` Because the result is a plain URL, `curl` is the object-verb surface: ```bash URL=$(lakeshore storage presign my-s3 hello.txt --put --expires-in 600) curl -X PUT --upload-file ./hello.txt "$URL" curl -s "$(lakeshore storage presign my-s3 hello.txt)" ``` ### `storage credentials` — temporary STS credentials Short-lived credentials a local script or SDK can use against the bucket directly. `--duration` defaults to 3600 seconds and accepts 900–129600. ```bash lakeshore storage credentials my-s3 lakeshore storage credentials my-s3 --duration 7200 --json eval "$(lakeshore storage credentials my-s3 --env)" # export into this shell ``` ## Code `lakeshore code` archives the current **git tree** and uploads it to a storage entry, so a remote daemon can pull the exact snapshot. ### `code push` — upload a snapshot Deduped by `(git remote, commit)` — re-running on the same clean commit is a no-op that reports `already archived`. The default storage entry name is the literal `code-staging`. ```bash lakeshore code push lakeshore code push --storage my-s3 ``` An uncommitted working tree is refused (exit 2) unless you pass `--dirty`, which stashes, archives, and pops. ```bash lakeshore code push --dirty ``` If the storage entry doesn't exist yet, the failure message tells you how to create it: ```bash lakeshore storage add code-staging --kind s3 \ --kwarg bucket= --kwarg 'creds.$secret=' ``` ### `code list` — show pushed snapshots ```bash lakeshore code list lakeshore code list --remote https://github.com/you/your-repo.git lakeshore code list --json ``` ## Mounts A **mount** is a declaration the runner attaches into the job workdir. The CLI writes metadata; the daemon does the actual mounting. `--kind` is required and must be one of: ```text nfs samba ftp sftp s3 s3fs google_drive dropbox bind configmap ``` > **Warning:** The `mounts add --kind` validator in CLI v0.2.0 accepts exactly the ten > kinds above. If you see a `git` kind documented elsewhere, it is ahead > of the shipped CLI. Each kind has its own required config fields, validated server-side: | Kind | Required | Also accepted | | -------------- | ------------------- | ---------------------------------------- | | `nfs` | `server`, `path` | — | | `samba` | `server`, `share` | path within the share, credentials | | `ftp` / `sftp` | `server` | path, credentials | | `s3` | `bucket` | `endpoint`, `region`, `prefix`, `creds` | | `s3fs` | `bucket` | `endpoint`, `region`, `prefix`, `creds`, `options` | | `google_drive` | `folderId` | `creds` | | `dropbox` | `path` | `creds` | | `bind` | `hostPath` | `readOnly` | | `configmap` | `kubeNamespace`, `name` | `items` | `s3` is the userspace-client form (copy on read); `s3fs` is the FUSE driver that gives job code a real filesystem path. ### `mounts add` ```bash # NFS export lakeshore mounts add datasets \ --kind nfs \ --kwarg server=10.0.0.10 \ --kwarg path=/srv/datasets # S3 via s3fs; creds.$secret references a registered secret by name lakeshore mounts add s3-cache \ --kind s3fs \ --kwarg bucket=my-bucket \ --kwarg prefix=cache/ \ --kwarg 'creds.$secret=s3-creds' # Host bind lakeshore mounts add hf-cache \ --kind bind \ --kwarg hostPath=/home/ge/.cache/huggingface \ --kwarg readOnly=true # Kubernetes ConfigMap lakeshore mounts add my-cm \ --kind configmap \ --kwarg kubeNamespace=default \ --kwarg name=app-cm ``` `--config-file` supplies the same fields from YAML or JSON. `--kind` stays on the command line either way: ```bash cat > /tmp/mount.yaml <<'YAML' server: 10.0.0.10 path: /srv/shared YAML lakeshore mounts add shared --kind nfs --config-file /tmp/mount.yaml ``` Any `$secret` marker inside the config must name a secret that exists in the same namespace, or the write is rejected with a 422. ### `mounts list` / `show` / `update` / `remove` ```bash lakeshore mounts list lakeshore mounts list --json lakeshore mounts show datasets --json lakeshore mounts update datasets --kwarg path=/srv/datasets-v2 lakeshore mounts remove datasets ``` ## Read next - [Storages](/get-started/storages.md) · [Mounts](/get-started/mounts.md) - [Payloads](/get-started/payloads.md) — how job inputs and outputs move. - [Secrets, modes, tunnels](/cli/examples/secrets-modes-tunnels-config-nymph.md) — registering the credentials these reference.