Kubernetes Agent Sandbox: Self-Hosted Setup on Your Cluster

The project is a SIG Apps subproject under Apache-2.0. Its core object, the Sandbox, describes an isolated, stateful, singleton workload: one Pod with a stable hostname, optional persistent storage through volumeClaimTemplates, an optional headless Service, and lifecycle controls. Those controls are shutdownTime (absolute expiry; the Pod and Service are always deleted when it passes), shutdownPolicy (Retain by default, or Delete, governing only the Sandbox object itself), operatingMode (Running or Suspended, where Suspended terminates the Pod and keeps the object and its volumes) and, new in v1.0.8, failedPodPolicy: Recreate, which replaces a Failed Pod while keeping the Sandbox identity and its PVCs.

Three extension objects turn that into something a platform team can offer as a service. They live in a second API group and only run when the controller starts with --extensions.

What it does not do matters as much. It does not isolate: that is the RuntimeClass. It does not give you per-claim network policy: rules live on the template, and every sandbox from that template shares one policy. It does not filter at layer 7: NetworkPolicy is L3/L4, so "allow this mirror on 443" is the finest grain you get without a CNI that does more. The v1alpha1 APIs and the conversion webhook were removed in v1.0.0 on 28 August 2026, so any tutorial that pauses a sandbox with replicas: 0 is teaching an API that no longer exists; v1beta1 uses operatingMode: Suspended.

Google Cloud's agent substrate is a separate GKE product layer and out of scope for a self-managed cluster.

The runtime is the isolation boundary, so choose it first. The project's own design material lists three options.

Our recommendation is Kata for anything an LLM wrote. That is consistent with the position we took in our comparison of why a microVM boundary beats gVisor for model-written code: a separate guest kernel behind a hypervisor is a stronger boundary than a user-space kernel, and gVisor's syscall interception costs the most on syscall-heavy code. We are not re-running that comparison here. If you only need to contain a coding agent on one developer's laptop, a local sandbox without a VM is the lighter answer; this post is about shared clusters.

The project's own evidence leans gVisor: its examples use it, its admission policy example mandates it, and gVisor is the only runtime with a regression job at v1.0.8 (a daily KIND job running the TestRuntimeClass* tests). Kata CI is still a proposal. The controller has no runtime-specific code paths, so Kata works through the same RuntimeClass mechanism, but a Kata regression would surface in your cluster first. Budget a staging run for every upgrade.

Two node-level notes for bare metal and private virtualization. If your nodes are themselves VMs, check that the hypervisor exposes hardware virtualization to them before you plan a Kata node pool. And use whatever class name kubectl get runtimeclass shows after kata-deploy runs; the project's quickstart uses kata-qemu.

Then enforce it. A SandboxTemplate pins runtimeClassName for every claim-provisioned sandbox. Bare Sandbox objects bypass templates, so they need admission control. The repo ships examples/policy/vap/secure-sandbox-policy.yaml, a ValidatingAdmissionPolicy whose expression requires object.spec.podTemplate.spec.runtimeClassName == 'gvisor'. Change that string to your Kata class name before you apply it, and remove or rewrite its last two rules, which require the GKE sandbox.gke.io/runtime: gvisor node selector and toleration. The policy matches every Sandbox, including those the claim controller creates from your templates, so templates must pass its other checks too, such as dropping ALL capabilities and setting CPU and memory limits on every container.

v1.0.8 publishes three manifests as release assets: sandbox.yaml (core only), extensions.yaml and sandbox-with-extensions.yaml. Use the last one. It runs the agent-sandbox-controller Deployment with --leader-elect=true and --extensions, and the image is registry.k8s.io/agent-sandbox/agent-sandbox-controller:v1.0.8. On an air-gapped cluster, download the manifest on a connected host, mirror that image into your registry and rewrite the image line before you apply.

export VERSION="v1.0.8"

# Air-gapped: download this file on a connected host, mirror
# registry.k8s.io/agent-sandbox/agent-sandbox-controller:v1.0.8
# into your registry, and rewrite the image line before applying.
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${VERSION}/sandbox-with-extensions.yaml

kubectl get crd sandboxes.agents.x-k8s.io
kubectl get deploy agent-sandbox-controller -n agent-sandbox-system

Deleting the CRDs cascade-deletes every custom resource of those types, so treat CRD removal as data deletion. A Helm chart exists under helm/; if you use it, apply the CRDs from helm/crds/ before helm upgrade, because Helm does not upgrade CRDs.

# Step 2: every line must print ["v1beta1"] and nothing else
kubectl get crd sandboxes.agents.x-k8s.io \
  sandboxclaims.extensions.agents.x-k8s.io \
  sandboxtemplates.extensions.agents.x-k8s.io \
  sandboxwarmpools.extensions.agents.x-k8s.io \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.storedVersions}{"\n"}{end}'

# Step 4: namespaced leftovers only
kubectl delete -n agent-sandbox-system \
  svc/agent-sandbox-webhook-service \
  secret/agent-sandbox-webhook-certs \
  role/agent-sandbox-controller \
  rolebinding/agent-sandbox-controller

The platform team owns the SandboxTemplate and the SandboxWarmPool. The agent runtime creates SandboxClaims against the pool, either as YAML or through an SDK. A claim must live in the same namespace as the pool it references.

The warm pool's replicas defaults to 1 and the CRD has a scale subresource, so an HPA can drive it. updateStrategy.type decides what happens when you edit the template: OnReplenish (the default) leaves stale unclaimed sandboxes in place until they are claimed or deleted, while Recreate deletes stale unclaimed sandboxes immediately. Claimed sandboxes are never replaced; ownership transfers to the claim. Use Recreate in a regulated setting: when you swap in a patched runtime image, idle sandboxes should be rebuilt now. It reacts only to changes in the template's Pod blueprint; an egress rule edit needs no rebuild, because it updates the shared NetworkPolicy in place.

Sizing is a trade between idle cost and claim latency. Every warm replica is a running Pod holding its resource requests, and under Kata also a running VM. Size the pool to the peak number of concurrent claims you need to serve without a cold start, not to daily volume. This matters more under Kata: the project's CI proposal estimates runc cold starts at about 0.3 seconds and Kata-class runtimes at 30 to 40 times longer, so a cold Kata claim waits roughly 9 to 12 seconds. That is a design-document estimate, not a benchmark, but it is the order of magnitude to plan around.

One rule defeats the pool silently: a claim that sets spec.env or spec.volumeClaimTemplates always cold-starts, even when ready sandboxes are waiting. additionalPodMetadata does not. The template fields envVarsInjectionPolicy and volumeClaimTemplatesPolicy both default to Disallowed; keep them there unless a specific workload needs per-claim configuration and can accept the cold start. The controller's refill-shaping flags matter only at high claim rates; the project suggests sharding pools above roughly 30 claims per second.

Claims expire on your terms: lifecycle.shutdownTime is an absolute timestamp enforced by the claim controller, and shutdownPolicy: Retain (the default) deletes the Sandbox, Pod and Service but keeps the claim object with an Expired status. Delete and DeleteForeground remove the claim too.

The manifest set below implements the on-prem posture: a Kata runtime class, no service-account token, dropped capabilities, resource limits, an explicit egress allowlist, a pool that rebuilds on template change and a claim with an expiry. The egress rules allow cluster DNS, which the default blocks. Know the trade: the sandbox can enumerate internal service names, and if CoreDNS forwards external queries, lookups become a low-bandwidth exfiltration channel. Connections are still denied except to one internal CIDR on 443. For IP-only egress, delete the first egress rule.

apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxTemplate
metadata:
  name: agent-exec-kata
  namespace: agent-sandboxes
spec:
  podTemplate:
    spec:
      runtimeClassName: kata-qemu        # whatever `kubectl get runtimeclass` shows
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      containers:
      - name: runtime
        image: registry.internal.example/agent-sandbox/python-runtime:2026-10
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
        ports:
        - containerPort: 8888
          protocol: TCP
        resources:
          requests:
            cpu: 500m
            memory: 1Gi
          limits:
            cpu: "2"
            memory: 2Gi
  failedPodPolicy: Retain
  envVarsInjectionPolicy: Disallowed
  networkPolicyManagement: Managed
  networkPolicy:
    ingress:
    - from:
      - namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: agent-sandbox-system
        podSelector:
          matchLabels:
            app: sandbox-router
      ports:
      - protocol: TCP
        port: 8888
    egress:
    - to:
      - namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: kube-system
      ports:
      - protocol: UDP
        port: 53
      - protocol: TCP
        port: 53
    - to:
      - ipBlock:
          cidr: 10.40.12.0/24             # internal package mirror (placeholder)
      ports:
      - protocol: TCP
        port: 443
---
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
  name: agent-exec-kata
  namespace: agent-sandboxes
spec:
  replicas: 4
  sandboxTemplateRef:
    name: agent-exec-kata
  updateStrategy:
    type: Recreate
---
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
  name: review-job-1842
  namespace: agent-sandboxes
spec:
  warmPoolRef:
    name: agent-exec-kata
  lifecycle:
    shutdownTime: "2026-10-23T18:00:00Z"
    shutdownPolicy: Retain               # keeps the claim as a record; Sandbox and Pod are deleted

The image, CIDR, namespace, user id and shutdownTime are placeholders, and replicas: 4 is illustrative. The runtime image has to serve the API the SDK calls (a POST to /execute on port 8888); the repo's examples/python-runtime-sandbox is a FastAPI server that does this, and you should build that image yourself, scan it and push it to your registry. If your CNI drops kubelet probe traffic under this policy, add an ingress rule for it.

This is where on-prem changes the answer most. When a template is Managed and you omit networkPolicy, the controller builds a default policy, one per template, named <template>-network-policy. It allows ingress only from pods labeled app: sandbox-router in agent-sandbox-system. For egress, it allows 0.0.0.0/0 and ::/0 with no port restriction, except 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, fc00::/7 and fe80::/10. The code comment says it plainly: the rule "allows all ports to PUBLIC IPs", and it "intentionally blocks internal cluster DNS". To compensate, when dnsPolicy is unset the controller sets dnsPolicy: None with nameservers 8.8.8.8 and 1.1.1.1.

That suits a cloud cluster. In a regulated perimeter it fails three ways.

The fixes are all template fields.

Four mechanics to know before you edit. A custom networkPolicy replaces the default entirely, so you must re-add the router ingress rule yourself, as the manifest above does. An empty egress: [] list blocks all egress, and the project's own air-gapped example sets both lists empty. Setting dnsPolicy: ClusterFirst explicitly also disables the DNS override. And because one policy serves every sandbox from a template, editing the template's rules updates the shared policy and your CNI applies it to existing and future sandboxes; a NetworkPolicy is only as good as the CNI enforcing it, so confirm yours does.

The router is an HTTP reverse proxy from SDKs and external clients to sandbox Pods, useful where, in the README's words, "direct pod port-forwarding is unavailable", as under Kata and gVisor. It is not mandatory: the Python SDK's in-cluster mode dials the Pod IP directly. Clients address a sandbox with headers: X-Sandbox-ID (required), X-Sandbox-Namespace and X-Sandbox-Port (default 8888). The router strips Host and Authorization before forwarding.

v1.0.8 changed the install. sandbox-router/deploy/ now contains a kustomization.yaml, so kubectl apply -f on the directory fails and kubectl apply -k is the command. The router's API token is now a projected token that expires after one hour. rbac-tokenreview.yaml is no longer applied by default, and an older install leaves its sandbox-router-auth-delegator ClusterRoleBinding behind because kubectl apply does not remove it.

git clone --depth 1 --branch v1.0.8 https://github.com/kubernetes-sigs/agent-sandbox.git

# Pin sandbox-router-go to a real tag in deployment.yaml before applying
kubectl apply -k agent-sandbox/sandbox-router/deploy/

# Only if the router runs --authz-mode=tokenreview:
kubectl apply -f agent-sandbox/sandbox-router/deploy/rbac-tokenreview.yaml

# Upgrading an older -f install that does NOT use tokenreview:
kubectl delete clusterrolebinding sandbox-router-auth-delegator

That block alone gives you a plain-HTTP, allow-all router running registry.k8s.io/agent-sandbox/sandbox-router-go:latest. The deploy README has its own "things to change before production" list, and on-prem every item applies.

Pick tokenreview when your agent services are the only callers and namespace boundaries already separate tenants. Pick scoped-token v2 when one router fronts sandboxes for several teams. Neither makes the request body safe: a holder of a scoped token can change the command it posts, so command policy belongs in the upstream service. For agent identity beyond router tokens, our piece on zero-trust controls for AI agents goes further.

TLS is opt-in, and the shipped deployment.yaml already carries every line you need as comments. Put a certificate from your internal CA in a Secret named sandbox-router-tls, uncomment the tls-certs volume and its /tls mount and the proxy-tls container port, and add these lines to the existing args list (keep --cache-enabled=true, which scoped tokens require):

- --https-bind-address=:8443
        - --tls-cert-file=/tls/tls.crt
        - --tls-key-file=/tls/tls.key
        - --authz-mode=tokenreview
        - --authz-tokenreview-require-token=true

Then uncomment the proxy-tls port 8443 in service.yaml and the 8443 ingress port in networkpolicy.yaml. The plain-HTTP listener on 8080 stays up until you set --http-bind-address= to an empty value. Without --authz-tokenreview-require-token=true, tokenreview mode forwards requests that carry no token at all. For client certificates, the router also supports --tls-client-ca-file and --mtls-mode. Two more items from the deploy README: tighten the example router NetworkPolicy's namespaceSelector: {} ingress to the namespaces your agents run in, and restrict Pod creation in agent-sandbox-system, because anyone who can create Pods there can run as the router and inherit its cluster-wide Pod read grant. Put rate limiting at your ingress tier too: the Go router sets no connection limits or timeouts for upgraded connections.

The repository supports two shapes, and the choice decides where credentials live. In the first, the whole agent process runs inside a Sandbox: examples/mcp-server-sandbox runs an MCP server in a bare Sandbox with a 1Gi PVC, suspends it with operatingMode: Suspended and resumes it with the volume reattached. In the second, the agent runs as a normal service and calls a sandbox to execute code, through the Python SDK or the MCP server's execute_command.

For regulated shops our default is sandbox as a tool. The agent that talks to the model and reads the documents keeps its credentials in a service you already know how to audit, and the box that runs generated code can have near-zero egress. Agents run code at all because executing code beats round-tripping data through tool calls; that same code is why it needs a box.

The Python SDK is pip install k8s-agent-sandbox==1.0.8 (Python 3.11 or later). This block runs from an in-cluster agent service through the TLS router configured above, authenticating with the service's own projected token; the 8443 port only answers after you enable --https-bind-address and the proxy-tls Service port, and the CA path is a placeholder.

from k8s_agent_sandbox import SandboxClient
from k8s_agent_sandbox.models import SandboxDirectConnectionConfig

with open("/var/run/secrets/kubernetes.io/serviceaccount/token") as f:
    service_account_token = f.read().strip()

client = SandboxClient(
    connection_config=SandboxDirectConnectionConfig(
        api_url="https://sandbox-router-svc.agent-sandbox-system.svc.cluster.local:8443",
        extra_headers={"Authorization": f"Bearer {service_account_token}"},
        ca_cert="/etc/pki/internal-ca.pem",
    )
)

sandbox = client.create_sandbox(
    warmpool="agent-exec-kata",
    namespace="agent-sandboxes",
    shutdown_after_seconds=900,
)
try:
    result = sandbox.commands.run("python3 /workspace/reconcile.py", timeout=120)
    print(result.exit_code, result.stdout)
finally:
    sandbox.terminate()

create_sandbox creates a SandboxClaim against the pool and waits for it, and shutdown_after_seconds is the backstop if the agent crashes before terminate. Go and TypeScript SDKs exist too.

If your agents use the project's MCP server instead (source-only in clients/integrations/mcp-server, not on PyPI), v1.0.8 changed it with no deprecation period: execute_command caps combined stdout and stderr at 30,000 Unicode code points by default, returns head-and-tail excerpts beyond that, accepts max_output_chars from 128 to 300,000, and adds four required response fields (stdout_total_chars, stderr_total_chars, stdout_truncated, stderr_truncated). Broader hardening of that server belongs in an MCP server security checklist.

Self-hosting is the reason to use this project. A hosted code-execution sandbox is faster to adopt, but the generated code and the documents it reads leave your perimeter. With agent-sandbox they stay on nodes you own, which is the same trade we lay out in cloud versus on-premise AI for regulated workloads.

The audit story comes from Kubernetes. Claims and sandboxes are API objects, so with API server auditing on, every creation and deletion is logged with the identity that made it. shutdownPolicy: Retain keeps each YAML-created claim after its Sandbox and Pod are gone, leaving a record per execution environment. The Python SDK's shutdown_after_seconds sets Delete and terminate() deletes the claim, so on the SDK path the API server audit log is the record. The controller emits Kubernetes Events unless you pass --disable-sandbox-events or --disable-claim-events, so do not. It exposes agent_sandbox_creation_latency_ms, and the router exports sandbox_router_authz_decisions_total with per-request access logs on by default.

Images are the other half. Mirror and pin three of them: the controller, the router (which ships as :latest) and the runtime image you built, which executes model-written code and gets the same scanning and signing as anything else you run.

What to do this week, on a non-production cluster:

For more on running agents inside your perimeter, start from our AI agents pillar.