Services

A Service is a long-running container that receives HTTP traffic. Use one for model inference APIs, dashboards, internal web tools, and other HTTP processes that should remain available instead of exiting after a finite task.

VM-backed Services on providers with public ingress receive a stable URL:

https://<deployment-id>.anycloud.sh

Use a Job for training, batch inference, or other work that should complete and automatically release its VM. Use a VM for direct SSH access without a managed public HTTP endpoint.

Start a service

The process inside the container must listen on 0.0.0.0 at the port in $PORT. The default is 8088.

CLI

anycloud service ghcr.io/acme/model-api:latest \
  --id model-api \
  --credentials my-aws \
  --gpu-type l40s \
  -- python -m myapp

The service becomes available at https://model-api.anycloud.sh.

Python

Use the generated admission operation for Services. Client.submit() creates Jobs, so the request below sets deploymentType explicitly:

from anycloud import Client
from anycloud.api.deployment_admission_api import DeploymentAdmissionApi
from anycloud.models.deployment_admission_input import DeploymentAdmissionInput

with Client() as client:
    service = DeploymentAdmissionApi(client.api_client).submit_deployment_sync(
        client.sdk_version,
        DeploymentAdmissionInput.from_dict({
            "id": "model-api",
            "image": "ghcr.io/acme/model-api:latest",
            "deploymentType": "service",
            "gpuType": "l40s",
            "command": ["python", "-m", "myapp"],
            "credentialName": "my-aws",
            "cloudConfig": {},
        }),
    )
    print(service.url)

Override the listening port with --env PORT=9000 or "env": {"PORT": "9000"} in the Python request. For a VM-backed Service, the public URL still routes to the configured port.

Lifecycle

Service lifecycleHappy path

Wait

⏳queued

Waiting for capacity or for a spend control to clear

→

Setup

🏗️provisioning

Create backing capacity

↓

⚙️initializing

Install the host runtime

↓

📥downloading

Pull the container image

↓

🚀starting

Start and check the service

→

Serve

⚡running

The service is live at its URL

Return paths

Setup failure→🛠️retrying↩⏳ queued

Upgrade→🔁recovering↩⏳ queued

Terminal exits

❌failedRequired retries exhausted

🚫invalidConfiguration rejected

🪦terminatedStopped by a user

A Service stops at running instead of completing. It remains there until you terminate it or start an upgrade. An upgrade enters recovering, replaces the backing deployment, and returns through queued while preserving the Service ID and, when present, public URL. This replacement has downtime while the old workload stops and the replacement becomes ready.

Replacement waits for the provider to confirm release of the old VM. Pending cleanup keeps the Service in recovering without consuming setup retry attempts.

Service behavior

• Services use on-demand VM capacity. Spot is rejected because an HTTP endpoint cannot use the Job checkpoint-and-restart contract.
• Job input/output buckets do not apply.
• Services can attach one pre-existing checkpoint bucket at /mnt/checkpoint. The Service must use a selected compute credential, and that same identity must have read-write bucket access. Local Services do not support it yet.
• Services use the same environment, secret, placement, disk, and Docker options as VM-backed Jobs.
• A service stays live until you terminate or upgrade it. It does not automatically shut down when a request completes.
• On VM-backed providers with public ingress, Anycloud exposes the application through its Anycloud URL. Your process should bind inside the container, not attempt to manage the URL.

Upgrade in place

Upgrade replaces the backing deployment while preserving its ID and, when present, public URL:

anycloud service upgrade model-api ghcr.io/acme/model-api:v2 \
  -- python -m myapp

Use an immutable image tag or digest when you need an auditable rollout. See service reference for the complete upgrade syntax.

With --checkpoint-bucket, Anycloud stops the old workload and completes a final checkpoint upload before removing its VM. The replacement restores that bucket before starting the new container. If the final upload fails, the old VM is retained for retry instead of continuing with stale state.

Operate a service

Services share the common deployment commands:

anycloud status model-api --watch
anycloud logs model-api
anycloud exec model-api "nvidia-smi"
anycloud terminate model-api

Termination safely flushes a configured checkpoint before removing the backing VM. The terminate command schedules cleanup and returns immediately.

Python uses the generated DeploymentsApi status and termination operations. To inspect an existing Service:

from anycloud import Client
from anycloud.api.deployments_api import DeploymentsApi

with Client() as client:
    deployments = DeploymentsApi(client.api_client)
    status = deployments.get_deployment_status_sync("model-api", client.sdk_version)
    print(status.deployment.state)

To terminate it:

from anycloud import Client
from anycloud.api.deployments_api import DeploymentsApi

with Client() as client:
    DeploymentsApi(client.api_client).terminate_deployment_sync(
        "model-api", client.sdk_version
    )

Termination schedules cleanup and returns immediately. Use the CLI for live logs and exec; the Python SDK does not include those native transports.