CLI examples — Providers and discover helpers

Part of the CLI examples cookbook. Auth setup lives on Auth + setup.

Providers

A provider is a registered cloud, SLURM, Kube, or SSH target. Add → list → test → launch is the happy path.

The provider name is a positional argument matching ^[a-z0-9][a-z0-9-]*$, and the launcher is --launcher — one of SSH, SLURM, EC2, GCE, Kube. --launcher is required unless you pass one of the --from-* flags, which imply it.

Not `--name` / `--type`

The lakeshore README's quick start shows providers add --name my-aws --type aws. That signature does not exist. Neither --name nor --type is a flag on providers add.

`providers add` — register an EC2 provider by hand

Inline kwargs. Dotted keys nest (so aws_credentials.accessKeyId becomes the nested object the server expects).

bash

lakeshore providers add my-ec2 \
  --launcher EC2 \
  --kwarg region=us-east-1 \
  --kwarg instance_type=t3.medium \
  --kwarg aws_credentials.accessKeyId=AKIAEXAMPLE \
  --kwarg aws_credentials.secretAccessKey=secret-example

`providers add` — from a local AWS profile

Pre-fills aws_credentials and region from the named ~/.aws profile. Implies --launcher EC2.

bash

lakeshore providers add ec2-prod --from-aws-profile prod

`providers add` — from a gcloud configuration

Pre-fills project_id, region, zone, and a service-account JSON from the named gcloud configuration. Implies --launcher GCE.

bash

lakeshore providers add gce-prod --from-gcloud-config prod

`providers add` — load kwargs from a file

For longer or nested configs. YAML or JSON; the file's top level becomes the provider's kwargs object.

bash

cat > /tmp/kube-kwargs.yaml <<'YAML'
context: my-kube-context
namespace: lakeshore-jobs
image: ghcr.io/dreamlake-ai/runner:latest
YAML
lakeshore providers add my-kube --launcher Kube --kwargs-file /tmp/kube-kwargs.yaml

`providers add --dispatch` — direct vs daemon

--dispatch is direct or daemon. Omit it and the launcher's default applies:

Launcher	Default dispatch
`SSH`	`direct`
`Kube`	`direct`
`SLURM`	`direct`
`EC2`	`daemon`
`GCE`	`daemon`

A dispatch key inside the provider's kwargs overrides the default too.

bash

lakeshore providers add my-ec2 --launcher EC2 --dispatch direct --kwarg region=us-east-1

`providers list` — table view

The default management view: everything registered in this namespace, excluding hidden and soft-deleted rows.

bash

lakeshore providers list
lakeshore providers list --json | jq -r '.[].name'
lakeshore providers list --hidden          # include hidden
lakeshore providers list --deleted         # include soft-deleted
lakeshore providers list --all             # both

Soft-deleted rows come back marked [D <ts>].

`providers show` — pretty-print one

Dumps the full kwargs for a single provider. Takes a name and no flags.

bash

lakeshore providers show my-ec2

`providers update` — merge in new kwargs

--kwarg deep-merges (repeatable); =null deletes a key. --kwargs-file replaces the kwargs object entirely.

bash

lakeshore providers update my-ec2 \
  --kwarg instance_type=t3.large \
  --kwarg root_volume_size_gb=100

lakeshore providers update my-ec2 --kwarg root_volume_size_gb=null

`--launcher` is immutable on update

Supplying --launcher to providers update is an error, not a no-op. Remove and re-add the provider to change its launcher.

`providers update --tunnel` — attach a tunnel

Bind a previously registered WireGuard tunnel to this provider. Pass --tunnel "" to clear.

bash

lakeshore providers update my-ec2 --tunnel wg-vpc
lakeshore providers update my-ec2 --tunnel ""

`providers test` — smoke-test the launch path

Runs a small Python script on the provider end-to-end. With no --script it uses the bundled stdlib-only hello.py, which prints the hostname, Python version, platform, and whether torch sees a GPU. --timeout defaults to 600 seconds.

bash

lakeshore providers test my-ec2
lakeshore providers test my-ec2 --script train.py --timeout 1200

SSH providers run directly from your machine. EC2, GCE, and Kube are server-mediated — the CLI POSTs to the control plane, which holds the cloud SDKs and credentials. SLURM prints a stub message; that path is not wired yet.

`providers instances` — live instance state

Hits the provider's launcher to list currently-running instances. --timeout defaults to 10 seconds. The name positional is optional when you pass --all.

bash

lakeshore providers instances my-ec2
lakeshore providers instances --all --timeout 15

`providers terminate` — kill a cloud instance

Two required positionals: provider then instance id. EC2 / GCE only. This is a provider-side terminate, not a daemon kill.

bash

lakeshore providers terminate my-ec2 i-0abc1234

`providers edit` — interactive editor

Arrow-key through the provider list and hide / unhide / delete / restore. Takes no arguments or flags, and expects a TTY.

bash

lakeshore providers edit

`providers discover` — unified picker

Surfaces local AWS profiles, GCP configs, and SSH hosts that are not yet registered, lets you multi-select, and registers them in one go.

bash

lakeshore providers discover
lakeshore providers discover --filter prod              # substring pre-filter
lakeshore providers discover --include-existing-ssh     # don't hide already-registered SSH hosts
lakeshore providers discover --page-size 30             # default 15
lakeshore providers discover --yes --filter prod        # skip the confirm

`providers remove` — delete

Drops the provider row. The cloud instances it spawned are unaffected.

bash

lakeshore providers remove my-ec2

Discover helpers

Local-only sniffers that inspect your machine's config files. None of them touch the control plane or the cloud — they're inputs to providers add / providers discover. Each exposes exactly one subcommand, discover, with exactly one flag, --json.

bash

lakeshore aws discover      # ~/.aws/credentials + ~/.aws/config
lakeshore gcp discover      # `gcloud`, falling back to ~/.config/gcloud
lakeshore slurm discover    # /etc/slurm/slurm.conf + `sinfo`
lakeshore kube discover     # `kubectl config view`
lakeshore docker discover   # `docker context ls`

lakeshore aws discover --json

SSH

lakeshore ssh reads ~/.ssh/config and can register hosts as providers.

`ssh discover` — list hosts

--filter accepts all (default), new (not yet on the server), or existing. Anything else exits 2.

bash

lakeshore ssh discover
lakeshore ssh discover --filter new
lakeshore ssh discover --filter existing
lakeshore ssh discover --show-stripped        # show local-only directives that were dropped

`ssh upload` — register an SSH host as a provider

bash

lakeshore ssh upload my-server                      # single alias
lakeshore ssh upload my-server --with-key           # also upload the IdentityFile as a secret
lakeshore ssh upload my-server --name prod-box      # provider name (default: the alias)
lakeshore ssh upload --all                          # every host, fingerprint-dedup'd
lakeshore ssh upload                                # no alias → multi-select picker
lakeshore ssh upload my-server --interactive        # force the picker anyway

Precedence: --all (or the literal alias --all) wins and is non-interactive; otherwise --interactive or a missing alias opens the picker; otherwise it's a single-alias upload.

Parsed ~/.ssh/config values can be overridden per upload with --user, --host, --port, and --pem. --key-secret <name> names the uploaded key secret (default ssh-<alias>). --dispatch accepts direct or daemon and is silently ignored for any other value.

`ssh probe` — try to connect, record locally

The result is cached at ~/.config/dreamlake/ssh-status.json and is not sent to the server. --timeout is the ssh ConnectTimeout, default 5 seconds.

bash

lakeshore ssh probe my-server
lakeshore ssh probe my-server --timeout 15
lakeshore ssh probe --all      # the literal string '--all' as the alias probes every host

`ssh status` — show the local probe cache

--history defaults to the 3 most recent probes per provider.

bash

lakeshore ssh status
lakeshore ssh status --name my-server
lakeshore ssh status --history 5

CLI examples — Providers and discover helpers

Providers

providers add — register an EC2 provider by hand

providers add — from a local AWS profile

providers add — from a gcloud configuration

providers add — load kwargs from a file

providers add --dispatch — direct vs daemon

providers list — table view

providers show — pretty-print one

providers update — merge in new kwargs

providers update --tunnel — attach a tunnel

providers test — smoke-test the launch path

providers instances — live instance state

providers terminate — kill a cloud instance

providers edit — interactive editor

providers discover — unified picker

providers remove — delete

Discover helpers

SSH

ssh discover — list hosts

ssh upload — register an SSH host as a provider

ssh probe — try to connect, record locally

ssh status — show the local probe cache

Read next

`providers add` — register an EC2 provider by hand

`providers add` — from a local AWS profile

`providers add` — from a gcloud configuration

`providers add` — load kwargs from a file

`providers add --dispatch` — direct vs daemon

`providers list` — table view

`providers show` — pretty-print one

`providers update` — merge in new kwargs

`providers update --tunnel` — attach a tunnel

`providers test` — smoke-test the launch path

`providers instances` — live instance state

`providers terminate` — kill a cloud instance

`providers edit` — interactive editor

`providers discover` — unified picker

`providers remove` — delete

`ssh discover` — list hosts

`ssh upload` — register an SSH host as a provider

`ssh probe` — try to connect, record locally

`ssh status` — show the local probe cache