Sandboxing Claude across three iterations: Docker, mitm, Kubernetes pod — Wintersalmon

This entire ten-post blog series was drafted by Claude agents running inside a sandboxed pod, and the sandbox got rebuilt three times across April to make that safe. Each rebuild upgraded one axis — isolation, then observability, then portability — and never weakened the previous one. The agent on the keyboard holds a real GitHub PAT and a real kubectl token, so this is the capstone of the LLM-powered dev flow: the sandbox is the harness for the same agent that writes the posts.

TL;DR

Phase 01 (April 21): local Docker, read-only rootfs, dropped caps, view ClusterRole — the isolation axis.
Phase 02 (April 22): mitmproxy sidecar plus a host-side Bun credential proxy on :9090 — the observability axis.
Phase 03 (April 23): K8s pod in claude-sandbox with PVCs, SOPS-encrypted Secrets, and a NetworkPolicy — the portability axis.
Sidebar: notifier-api-go fans a single POST out to Discord and Telegram so I can walk away from long sessions.
One risk per phase, one rebuild per axis. Never collapse two phases into one.

The threat model the phases were built against

Three concrete fears shaped the rebuilds: token exfiltration through a normal-looking API call, accidental destruction (rm -rf, kubectl delete, git push --force), and unobservable network activity. Each phase tackled exactly one. Forcing the one-risk-per-phase mapping stopped me from sliding from “isolation” into “let’s also rewrite the secrets workflow” mid-build.

Phase 01 — Docker isolation cleared the easy axis first

A Debian 12 slim image with claude, gh, kubectl, bun, and tmux baked in. Two host bind mounts persist state: workspace/ for repo clones, home/ for the Claude OAuth token. Two bugs worth recording: tooling under /home/agent/.bun got shadowed by the empty home bind mount (fix: install to /opt), and the pre-seeded anthropic.key was Claude Code’s OAuth JSON not a raw API key (fix: drop the pre-seed, run claude login once inside).

The container flags are the load-bearing part:

docker run -d --name cn-claude-default \
  --user 1000:1000 --read-only --tmpfs /tmp:size=512m \
  --cap-drop=ALL --security-opt=no-new-privileges \
  --memory=4g --cpus=2 --pids-limit=512 \
  cn-claude-sandbox:phase01

PID 1 is tini calling tail -f /dev/null; tmux starts detached in the entrypoint. My first version used docker run --rm -it ... tmux new, and the moment I hit Ctrl-b d the tmux client exited, PID 1 followed, and the container died. The K8s side: a claude-readonly ServiceAccount bound to the built-in view ClusterRole — Secrets are excluded because RBAC has no verb that returns Secret metadata without .data.

Phase 02 — mitmproxy and a credential proxy made the network legible

After a week of Phase 01 I had no idea what URLs the agent was hitting. A mitmproxy sidecar joins the same Docker network; the sandbox sets HTTP_PROXY=http://mitm:8080, the mitm CA is baked into the trust store via update-ca-certificates, and NODE_EXTRA_CA_CERTS covers Node-based callers. Anthropic API, GitHub, K8s — all plaintext flows, viewable live with csb traffic <name>.

A gotcha: kubectl verifies against the cluster CA, not the system store, so its calls failed with TLS errors until I added the K8s server IP to NO_PROXY. The second move was a host-side credential proxy — a Bun HTTP server on localhost:9090 holding the GitHub PAT and kubeconfig, exposing a narrow surface (git/clone, gh/:path, k8s/:path). After removing the secrets mount, cat /run/secrets/github.token returns “no such file” and env | grep GH_TOKEN is empty. Validation suite: 23 checks, 23 passes.

Phase 03 — Kubernetes pod cut the laptop tether

Phase 02 still ran on my laptop. Sessions should survive reboots and “I’m leaving the house.” The pod runs the sandbox container and a mitmproxy/mitmproxy sidecar in one spec; two PVCs back /workspace and /home/agent, so a pod delete keeps data and a fresh pod resumes where the last one stopped. Credentials moved from ~/claude-sandbox/secrets/ to a SOPS-encrypted csb-secrets Secret reconciled by FluxCD. Every docker call in the csb CLI became a kubectl call — csb attach <name> is now kubectl exec -it csb-<name> -c sandbox -- tmux attach -t claude.

A NetworkPolicy enforces that the sandbox container can’t egress directly — only via the in-pod mitm sidecar on port 8080 and the host credential proxy on 9090:

spec:
  podSelector: { matchLabels: { app: csb-session } }
  policyTypes: [Egress]
  egress:
    - ports: [{ port: 8080 }]
    - to: [{ namespaceSelector: {} }]
      ports: [{ port: 9090 }]

Pod spec drafted, manifests staged, migration in progress.

notifier-api-go is a Go service in cloudnest-functions, ClusterIP only, at http://notifier-api-go-stag.cloudnest-functions.svc.cluster.local/api/v1/notify. Bearer-token auth from a SOPS-encrypted notifier-credentials Secret; one POST with title/message/level/source fans out to Discord and Telegram. 20 tests pass with -race; the sandbox pod gets NOTIFIER_URL and NOTIFIER_AUTH_TOKEN injected via secretKeyRef.

What this changes going forward

A long-running Claude session that writes real code, holds no credential I wouldn’t give a junior contractor on day one, lets me see every byte it sends, and isn’t chained to a laptop. This series — including this post — was drafted by an agent inside one of these sandboxes, then handed to the publishing pipeline written up in Building a Content-Publishing Workflow from Task Logs. Rule for next time: when adding a capability, name which phase’s invariant it strengthens; never weaken any of them silently.

AI workflow note

This design was planned by Claude planning its own sandbox, with a threat-model-first prompt pattern: write the three risks in plain prose before any Dockerfile or YAML, then map one risk per phase. That mapping is what kept scope from collapsing — every time the agent suggested merging “isolation” and “credential isolation,” the plain-prose threat list pushed back. The failure mode: asking Claude to “harden the sandbox” without naming the axis produced a generic checklist with no testable invariant. Repeated to each phase’s agent: never hand an agent a credential you wouldn’t hand a day-one junior contractor.

#claude #sandbox