Buckets

Buckets connect durable cloud object storage to a Job or VM-backed Service as ordinary files. Anycloud handles S3, Azure Blob Storage, or Google Cloud Storage access and bind-mounts the synchronized directories into the container.

Use Buckets for large datasets, model weights, results, logs, and state that must survive a spot interruption. They are a first-class platform primitive, not an image-building detail.

The three mounts

Role	Container path	Workloads	Access	Sync
Input	/mnt/input	Jobs	Read-only	Download once before startup
Output	/mnt/output	Jobs	Read-write	Upload about every 60 seconds and at exit
Checkpoint	/mnt/checkpoint	Jobs and VM-backed Services	Read-write	Restore at startup; upload about every 60 seconds and at exit

Input buckets must already exist. Output buckets are created when necessary. Input and output must use different bucket names. An explicitly named checkpoint bucket must already exist. It is user-owned: Anycloud syncs its contents but never creates or deletes the bucket. A spot Job without an explicit name keeps the existing behavior—Anycloud creates and manages a checkpoint bucket named after the deployment. Hosted APIs use the same explicit retained contract: create a dedicated bucket first and pass it to anycloud api serve --checkpoint-bucket.

Attach input and output

CLI

anycloud job ghcr.io/acme/train:latest \
  --credentials my-aws \
  --gpu-type a100:8 \
  --spot \
  --input-bucket training-data \
  --output-bucket training-results \
  -- python train.py

Python

from anycloud.api.deployment_admission_api import DeploymentAdmissionApi
from anycloud.models.deployment_admission_input import DeploymentAdmissionInput

job = DeploymentAdmissionApi(client).submit_deployment_sync(
    sdk_version,
    DeploymentAdmissionInput.from_dict({
        "image": "ghcr.io/acme/train:latest",
        "gpuType": "a100:8",
        "credentialName": "my-aws",
        "cloudConfig": {
            "inputBucket": "training-data",
            "outputBucket": "training-results",
        },
        "command": ["python", "train.py"],
    }),
)

Inside the container, application code only reads and writes paths:

from pathlib import Path

model = Path("/mnt/input/base-model.bin").read_bytes()
Path("/mnt/output/metrics.json").write_text('{"loss": 0.12}')

Attach checkpoint state

Create the bucket first, then attach the same name to a Job or VM-backed Service:

anycloud bucket create model-state --credentials my-aws
anycloud service ghcr.io/acme/model-api:latest \
  --credentials my-aws \
  --vm-type g5.xlarge \
  --checkpoint-bucket model-state

In Python, set cloudConfig.checkpointBucket in the generated deployment admission model. The process reads and writes /mnt/checkpoint.

For Services, the checkpoint Bucket must use the selected compute credential. Separate or cross-account checkpoint storage credentials are supported only for Jobs. The Bucket may be in another region when the same compute identity can access it.

Sync guarantees

Input uses rclone sync once before startup. Output uses upload-only rclone copy, so two Jobs writing to one bucket do not delete one another's objects. Checkpoints restore existing objects first and then use the same upload-only behavior.

Input hydration has no fixed wall-clock deadline: a Job remains in syncing until the whole bucket is present, however long a healthy transfer takes. Rclone emits activity once a minute; Anycloud stops and retries a worker only after 15 minutes without new output, and reattaches to an active VM-side sync after an API restart. Self-hosted API operators can tune that watchdog with INPUT_SYNC_INACTIVITY_TIMEOUT_MS.

An input mount always hydrates the entire bucket; it does not accept an object prefix. Startup time therefore scales with both bytes and object count. Use a dedicated input bucket for each pipeline stage when a shared bucket contains unrelated data.

When a bucket-backed Job exits, it enters finalizing while Anycloud performs one last output and checkpoint copy. completed or errored is published only after that succeeds. If final sync exhausts its retries, the Job becomes failed because its artifacts could not be preserved.

Before a VM-backed Service is terminated or replaced, Anycloud stops the workload, stops continuous checkpoint sync, completes one final upload, and only then removes the VM. A failed final upload leaves the VM registered so cleanup can retry instead of silently replacing it with stale state.

Periodic sync is file copying, not an application-aware database snapshot. A clean Service termination or replacement quiesces the process first, but sudden VM loss can lose roughly one sync interval, and the last live-file copy is not guaranteed to be a coherent multi-file snapshot. Applications with multi-file or hot database state should write their own atomic checkpoint into /mnt/checkpoint. Use one active writer per checkpoint bucket.

Files exist once on the VM and are exposed through Docker bind mounts; there is no second copy inside the container. The host paths can differ by provider, but the three /mnt/... container paths are stable.

Work with objects directly

Use the CLI when you need to discover, prepare, or inspect a bucket outside a Job. With no target, list enumerates buckets visible to the selected AWS, GCP, or Azure credential. With a target, it shows top-level objects and virtual folders:

anycloud bucket list --credentials my-aws
anycloud bucket create training-data --credentials my-aws
anycloud bucket upload training-data ./dataset.parquet data/dataset.parquet \
  --credentials my-aws
anycloud bucket list training-data --credentials my-aws
anycloud bucket list training-data data/ --credentials my-aws
anycloud bucket list training-data --recursive --credentials my-aws
anycloud bucket download training-data data/dataset.parquet ./dataset.parquet \
  --credentials my-aws
anycloud bucket upload training-data ./dataset-shards data/shards/ --recursive \
  --credentials my-aws
anycloud bucket download training-data data/shards/ ./dataset-shards --recursive \
  --credentials my-aws

Prefixes are literal; use a trailing / to browse a folder. Add --recursive for a flat full-tree view. Listings default to 1,000 sorted results; --all exhausts all pages.

Recursive upload and download preserve paths relative to the supplied literal folder prefix. Use '' as the prefix to copy the bucket root. Recursive prefix deletion requires a non-empty trailing-slash prefix and confirmation (or --yes for automation):

anycloud bucket rm training-data stale-runs/ --recursive --yes \
  --credentials my-aws

Directory operations default to four concurrent object requests and accept --concurrency 1 through 32. They preflight the complete manifest before starting, attempt every item after an individual failure, and report partial success without rollback.

The generated SDK exposes JSON bucket metadata operations through BucketsApi:

from anycloud.api.buckets_api import BucketsApi

buckets = BucketsApi(client)
page = buckets.list_buckets_sync("my-aws", sdk_version, page_size="100")
for bucket in page.buckets:
    print(bucket.to_dict()["name"])

objects = buckets.list_bucket_objects_sync(
    "my-aws", "training-data", sdk_version, prefix="data/"
)

Follow each response's next_cursor to paginate. Upload, download, and directory transfer are native byte transports and are not included in the generated SDK; use the CLI for those operations.

See bucket reference for exact commands.

Authentication

Same-cloud sync uses the VM's cloud-native identity:

• AWS IAM roles for S3
• Azure managed identities for Blob Storage
• GCP service accounts for Cloud Storage

Long-lived cloud credentials are not injected into the workload container.

Azure separates controller and workload authorization. The shared VM managed identity keeps its storage-account-scoped Storage Blob Data Contributor role and uses Entra RBAC for Blob access. The Anycloud controller keeps only its subscription-scoped Contributor and User Access Administrator roles. It uses Contributor's listKeys access to fetch a current account key for controller Blob operations. That key grants broad data-plane access but is never persisted or logged.

This controller model requires Allow storage account key access on the Anycloud-managed storage account. Anycloud explicitly enables it when creating the account and reports an actionable error if policy later disables it. See Microsoft's Shared Key security guidance when evaluating this tradeoff.

If storage belongs to a different account, set the corresponding generated admission fields such as inputStorageCredentialName and inputStorageRegion.

For Jobs, use the corresponding --input-storage-*, --output-storage-*, or --checkpoint-storage-* flags. The three identities are independent. Ordinary Service checkpoints are the exception: they require a selected compute credential and always use that compute identity. A hosted Anycloud API may use --checkpoint-storage-credentials with AWS S3 so its durable state can live outside a compute provider that has no compatible object storage.

Cross-cloud storage currently supports AWS S3. Anycloud mints bucket-scoped, short-lived STS credentials rather than copying the saved AWS key to the VM. The token lasts up to 36 hours and is not refreshed during periodic sync for a long-running workload. VM-hosted workloads mint a fresh token for their final copy so termination can still preserve accumulated output and checkpoint data. Worker-targeted Jobs do not support per-Job Buckets.

AWS, Azure, and GCP Jobs can use the selected compute identity for same-account Buckets, including automatic spot checkpoints. Lambda, Vast, and Local Jobs require explicit storage credentials and a storage region for checkpoint buckets because they do not provide a compatible native bucket identity. Scoped cross-identity storage currently supports AWS. Checkpointed ordinary Services always use their selected compute identity; hosted APIs have the explicit AWS state-credential exception above.