Skip to main content

Services

A Service is a long-running container that receives HTTP traffic. Use one for model inference APIs, dashboards, internal web tools, and other HTTP processes that should remain available instead of exiting after a finite task.

VM-backed Services on providers with public ingress receive a stable URL:

https://<deployment-id>.anycloud.sh

Use a Job for training, batch inference, or other work that should complete and automatically release its VM. Use a VM for direct SSH access without a managed public HTTP endpoint.

Start a service​

The process inside the container must listen on 0.0.0.0 at the port in $PORT. The default is 8088.

Pass --cluster <id-or-name> instead of VM placement flags to run the Service on a Ready cluster. Anycloud creates a native Kubernetes Deployment and cluster-internal Service, then stores the selected cluster so later lifecycle operations cannot drift to another one.

anycloud service ghcr.io/acme/model-api:latest \
  --id model-api \
  --credentials my-aws \
  --gpu-type l40s \
  -- python -m myapp

The service becomes available at https://model-api.anycloud.sh.

To run the same application on shared cluster capacity:

anycloud service ghcr.io/acme/model-api:latest \
  --id model-api \
  --cluster training \
  -- python -m myapp

Cluster-backed Services do not receive a public Anycloud URL yet.

Override the listening port with --env PORT=9000 or env={"PORT": "9000"}. For a VM-backed Service, the public URL still routes to the configured port.

Lifecycle​

Service lifecycleHappy path
Wait
queued
Waiting for capacity or for a spend control to clear
Setup
provisioning
Create backing capacity
initializing
Install the host runtime
downloading
Pull the container image
starting
Start and check the service
Serve
running
The service is live at its URL
Return paths
Setup failureretrying⏳ queued
Upgraderecovering⏳ queued
Terminal exits
failedRequired retries exhausted
invalidConfiguration rejected
terminatedStopped by a user

A Service stops at running instead of completing. It remains there until you terminate it or start an upgrade. An upgrade enters recovering, replaces the backing deployment, and returns through queued while preserving the Service ID and, when present, public URL. This replacement has downtime while the old workload stops and the replacement becomes ready.

Service behavior​

  • VM-backed Services use on-demand capacity; cluster-backed Services use shared cluster capacity. Spot is rejected because an HTTP endpoint cannot use the Job checkpoint-and-restart contract.
  • Job input/output buckets do not apply.
  • VM-backed Services can attach one pre-existing checkpoint bucket at /mnt/checkpoint. The Service must use a selected compute credential, and that same identity must have read-write bucket access. Local and Kubernetes-backed Services do not support it yet.
  • VM-backed Services use the same environment, secret, placement, disk, and Docker options as Jobs. Cluster-backed Services support environment values, named secrets, --gpus, and --shm-size; other VM-placement and Docker-runtime options fail at submission.
  • A service stays live until you terminate or upgrade it. It does not automatically shut down when a request completes.
  • On VM-backed providers with public ingress, Anycloud exposes the application through its Anycloud URL. Your process should bind inside the container, not attempt to manage the URL.

Upgrade in place​

Upgrade replaces the backing deployment while preserving its ID and, when present, public URL:

anycloud service upgrade model-api ghcr.io/acme/model-api:v2 \
  -- python -m myapp

Use an immutable image tag or digest when you need an auditable rollout. See service reference for the complete upgrade syntax.

With --checkpoint-bucket, Anycloud stops the old workload and completes a final checkpoint upload before removing its VM. The replacement restores that bucket before starting the new container. If the final upload fails, the old VM is retained for retry instead of continuing with stale state.

Operate a service​

VM-backed Services share the common deployment commands:

anycloud status model-api --watch
anycloud logs model-api
anycloud exec model-api "nvidia-smi"
anycloud terminate model-api

For a cluster-backed Service, status reports the persisted lifecycle state; live Pod health, logs, and exec are not yet supported. Upgrade and termination use the stored cluster placement.

For a VM-backed Service, termination safely flushes a configured checkpoint before removing the backing VM. The terminate command schedules cleanup and returns immediately.

The Python Service handle exposes state(), status(), logs(), exec(), and terminate(); support for live operations depends on the selected execution target. See Python SDK: Service Methods.