feat: buckets

skills/excloud-cli/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: excloud-cli
-description: Drive Excloud resources (compute, networking, security groups, volumes, snapshots, public IPs, IAM, billing, Kubernetes) through the `exc` CLI. Use when a user asks to plan or execute `exc` commands - creating / inspecting / updating / deleting VMs, running commands on them via `exec` / `scp` / `console`, managing security groups and public IPs, or pulling Kubernetes kubeconfigs - with safety guardrails and auth checks.
+description: Drive Excloud resources (compute, networking, security groups, volumes, snapshots, public IPs, IAM, billing, Kubernetes, object storage buckets) through the `exc` CLI. Use when a user asks to plan or execute `exc` commands - creating / inspecting / updating / deleting VMs, running commands on them via `exec` / `scp` / `console`, managing security groups and public IPs, pulling Kubernetes kubeconfigs, or managing buckets/objects/access keys - with safety guardrails and auth checks.
 ---
 
 # Excloud CLI
@@ -47,6 +47,8 @@ Require explicit user confirmation before running any of these:
 - `exc securitygroup delete`, `exc securitygroup rule delete`, `exc securitygroup binding delete`.
 - `exc k8s cluster delete`, `exc k8s cluster worker delete`.
 - `exc account revoke`, `exc serviceaccount delete`, `exc apikey delete`, `exc policy delete`, `exc policy binding delete`.
+- `exc buckets delete`, `exc buckets rm`, `exc buckets objects delete`, recursive bucket/object deletes, and `exc buckets multipart abort`.
+- `exc buckets keys delete`; also treat `exc buckets keys create` output as sensitive because the secret is shown once.
 
 For shell commands delivered through `exc compute exec` or an `exec` script file, refuse or confirm explicitly before running anything like `shutdown`, `reboot`, `rm -rf`, `mkfs`, `dd`, `wipefs`, rewrites of `/etc/fstab`, bootloader edits, or `systemctl stop ssh*` (the last one will make the VM unreachable over SSH — see Interactive access).
 
@@ -60,6 +62,9 @@ The skill does _not_ hard-code IDs, instance type names, image IDs, subnet IDs,
 - `exc compute subnet list` + `exc compute subnet get --id <id>` — check `DISABLE_IPV4_PUBLIC_IP`: subnets with this set cannot take `--allocate_public_ipv4=true` at create time.
 - `exc securitygroup list` + `exc securitygroup rule list --security_group_id <id>` + `exc securitygroup binding list --security_group_id <id>` (or `--interface_id <id>`) — confirm what a SG allows and where it's bound before relying on it.
 - `exc compute publicip list` / `exc compute key list` / `exc compute volume list` / `exc compute snapshot list` — authoritative inventories for each resource type.
+- `exc buckets list` / `exc buckets get <name>` / `exc buckets usage` — authoritative object-storage inventory and quota checks.
+- `exc buckets objects list <bucket> [--prefix <prefix>] [--all]` — discover keys before copy/delete/share operations.
+- `exc buckets keys list` and `exc buckets multipart list <bucket>` — discover S3 access keys and abandoned multipart uploads before managing them.
 
 If `--help` on the installed CLI shows commands or flags not documented here, prefer `--help`.
 
@@ -275,6 +280,69 @@ exc compute volume list             # the root volume should disappear / move to
 - `exc k8s cluster kubeconfig merge --cluster_id <id> [--kubeconfig <path>] [--backup=true|false]` — merges into `~/.kube/config` (or `--kubeconfig`) using `kubectl config view --merge --flatten --raw`. Requires `kubectl` on PATH. `--backup` defaults to `true` and writes `<path>.bak`, `<path>.bak1`, ... before overwriting.
 - `exc k8s bootstrap controlplane get --vm_id <id> --x-exc-imds-token <token>` — operator bootstrap path; the IMDS token must come from inside the VM's IMDS agent, not be invented.
 
+## Object storage buckets
+
+The buckets API endpoint is `https://buckets.excloud.dev`. The `exc buckets` commands are for Excloud object storage management; S3-compatible clients use bucket access keys configured through `exc buckets keys configure`.
+
+Always check the installed CLI first:
+
+```bash
+exc buckets --help
+exc buckets objects --help
+exc buckets keys --help
+exc buckets multipart --help
+```
+
+- `exc buckets list [--json]` — table of bucket name, public flag, object count, size, and creation time.
+- `exc buckets create <name> [--public]` — bucket names are lowercase letters, digits, and hyphens, 3-63 chars, no leading/trailing hyphen.
+- `exc buckets get <name>` — JSON bucket detail, including public URL when public.
+- `exc buckets update <name> --public=true|false`
+- `exc buckets usage` — JSON org-wide bucket/object/byte counters and quotas.
+- `exc buckets delete <name> [--force] [--yes] [--json]` — destructive. Without `--force`, expect the API to refuse non-empty buckets; with `--force`, the CLI empties the bucket via object listing and bulk delete before deleting. Confirm unless the user already authorised it.
+- `exc buckets ls` — lightweight bucket table with bucket name, size, and creation time.
+- `exc buckets ls <bucket[/prefix] | s3://bucket/prefix> [--all] [--json]` — object listing shortcut.
+
+Remote object references are command-specific. Commands that take a combined remote ref generally accept either `bucket/key` or `s3://bucket/key`; commands with explicit `<bucket> <key>` forms also accept those explicit args. Two important exceptions: `exc buckets cp` treats operands as remote only when they use `s3://bucket/key`, and `exc buckets objects list` only splits a combined single arg when it is `s3://bucket/prefix`; use `exc buckets objects list <bucket> [prefix]` or `--prefix` for the plain form.
+
+- `exc buckets objects list <bucket> [prefix] [--prefix <prefix>] [--cursor <token>] [--all] [--json]`
+- `exc buckets objects upload <bucket> <local-path> [--key <remote-key>] [--content-type <mime>] [--recursive]`
+  - Files over the single-upload cap are automatically uploaded using multipart.
+  - Directories require `--recursive`.
+  - If `--key` ends in `/`, the local basename is appended.
+- `exc buckets objects download <bucket> <key> [--output <path>]` — also accepts a single `bucket/key` or `s3://bucket/key` arg.
+- `exc buckets objects copy <src-bucket/key> <dst-bucket/key>` — server-side copy; no local data transfer.
+- `exc buckets objects delete <bucket> <key...> [--recursive] [--yes] [--json]` — destructive. `--recursive` expands the first key as a prefix and deletes matching objects in chunks.
+- `exc buckets objects presign <bucket> <key> [--ttl <seconds>]` — also accepts a single `bucket/key` or `s3://bucket/key` arg. Scriptable: prints only the URL.
+- `exc buckets objects share <bucket> <key> [--ttl <seconds>]` — also accepts a single `bucket/key` or `s3://bucket/key` arg. Prints the URL and expiry.
+- `exc buckets objects mkdir <bucket> <prefix>` — also accepts a single `bucket/prefix` or `s3://bucket/prefix` arg. Creates a virtual folder placeholder; appends `/` if missing.
+- `exc buckets objects sync <local-dir> <bucket/prefix> [--delete] [--dry-run] [--exclude <glob>] [--include <glob>]`
+  - Use `--dry-run` before a large sync, especially with `--delete`.
+  - The CLI compares size and ETag when possible. Multipart-style ETags may not be reproducible locally, so changed-vs-skipped decisions should be treated conservatively.
+- `exc buckets cp <src> <dst> [--recursive] [--content-type <mime>]`
+  - Remote operands must use `s3://bucket/key`; plain `bucket/key` is local for this command.
+  - local file → remote uploads one file.
+  - local directory → remote requires `--recursive`.
+  - remote → local downloads one object, or a prefix with `--recursive`.
+  - remote → remote uses server-side copy.
+  - local → local is a usage error; use normal shell tools.
+- `exc buckets rm <bucket/key | s3://bucket/key> [--recursive] [--yes] [--json]` — destructive shortcut for object delete.
+
+- `exc buckets keys list [--json]`
+- `exc buckets keys create <name>` — prints the `secret_access_key` once. Capture it immediately; the CLI cannot fetch it again later.
+- `exc buckets keys delete <akid>` — destructive, immediate revocation.
+- `exc buckets keys configure <akid> [--profile <name>] [--secret <secret>] [--set-default]`
+  - Writes `~/.aws/credentials` and `~/.aws/config` for S3-compatible access to `https://buckets.excloud.dev`.
+  - Prefer the interactive secret prompt or `EXCLOUD_BUCKETS_SECRET`; `--secret` is convenient but can land in shell history.
+  - Default AWS profile name is typically `excloud`. Test with `aws s3 ls --profile excloud`.
+
+Normal users should prefer `objects upload`; it automatically uses multipart for large files. Use `exc buckets multipart` for manual part control or abandoned-upload cleanup:
+
+- `exc buckets multipart list <bucket> [--prefix <prefix>] [--limit <n>] [--all] [--json]`
+- `exc buckets multipart create <bucket> <key> [--content-type <mime>]` — also accepts a single `bucket/key` or `s3://bucket/key` arg.
+- `exc buckets multipart uploadpart <bucket> <uploadId> <partN> <file>` — part numbers are 1-10000.
+- `exc buckets multipart complete <bucket> <uploadId> --parts '<json-array>'` — `--parts` can be inline JSON or a path to a JSON file.
+- `exc buckets multipart abort <bucket> <uploadId>` — destructive cleanup of an unfinished upload.
+
 ## IAM, billing, quota
 
 - `exc org list`
@@ -296,8 +364,10 @@ exc compute volume list             # the root volume should disappear / move to
 
 Every command either prints a column table (or TSV) or prints JSON — no command should print raw Go-struct dumps anymore. Both shapes are machine-parseable; pick your tool accordingly.
 
-- **Column tables / TSV** (awk / `cut` / `awk -F\t` friendly): `compute list`, `compute get`, `compute create`, `compute terminate` (TSV `vm_id\tstate`), `compute instancetype list` / `capacity`, `compute image list`, `compute subnet list`, `compute volume list`, `compute volume get`, `compute snapshot list`, `compute publicip list`, `compute key list`, `securitygroup list` / `rule list` / `binding list`, `org list`, `account list`, `apikey list`, `policy list`, `config list`, `compute seriallogs`.
-- **JSON** (pipe through `jq`): `me`, `quota`, `billing get`, `compute health` (`{"raw":"OK"}`), `k8s health`, `compute subnet get`, `compute publicip get`, `compute key get`, `securitygroup get`, `compute metrics`, `compute connect`, `serviceaccount list`, `compute protect`, `compute unprotect`, `compute rename`, `k8s cluster kubeconfig get` (raw kubeconfig YAML, not JSON-wrapped), and the inline `kubeconfig` field inside the JSON response from `k8s cluster create` when `-o` is not set.
+- **Column tables / TSV** (awk / `cut` / `awk -F\t` friendly): `compute list`, `compute get`, `compute create`, `compute terminate` (TSV `vm_id\tstate`), `compute instancetype list` / `capacity`, `compute image list`, `compute subnet list`, `compute volume list`, `compute volume get`, `compute snapshot list`, `compute publicip list`, `compute key list`, `securitygroup list` / `rule list` / `binding list`, `org list`, `account list`, `apikey list`, `policy list`, `config list`, `compute seriallogs`, `buckets list`, `buckets ls`, `buckets objects list`, `buckets keys list`, `buckets multipart list`.
+- **JSON** (pipe through `jq`): `me`, `quota`, `billing get`, `compute health` (`{"raw":"OK"}`), `k8s health`, `compute subnet get`, `compute publicip get`, `compute key get`, `securitygroup get`, `compute metrics`, `compute connect`, `serviceaccount list`, `compute protect`, `compute unprotect`, `compute rename`, `k8s cluster kubeconfig get` (raw kubeconfig YAML, not JSON-wrapped), `buckets get`, `buckets usage`, `buckets keys create`, multipart create/uploadpart/complete/abort, and the inline `kubeconfig` field inside the JSON response from `k8s cluster create` when `-o` is not set.
+
+Many bucket list/delete commands have explicit `--json`; use it for scripts instead of scraping tables. `buckets objects presign` intentionally prints only the URL so it can be piped to tools like `pbcopy`.
 
 Before scripting heavy logic against a command, run it once and check the shape. The split between "table" and "JSON" is not always guessable — lists tend to be tables, getters tend to be JSON, but verify.