Sandboxed Execution on Kubernetes: How the Platform Isolates AI Agent Tool Runs

When an AI agent generates and executes code, you have two options: trust it or sandbox it. Every production outage, data breach, and compliance violation caused by an AI agent can be traced back to choosing the first option.

The platform sandboxes every tool execution by default. Not optionally. Not as a configuration flag. Every tool — whether synthesized by OATS on demand or called through an MCP server — runs in an isolated environment with static validation, resource limits, and credential scoping. Here is how the sandbox architecture works.

Layer 1: Static Code Validation

Before any generated code reaches an execution environment, it passes through static validation. The validator blocks known-dangerous patterns:

• eval() and exec() — arbitrary code execution
• os.system() and subprocess calls — shell escape
• Dynamic imports — dependency injection attacks
• __import__ — runtime module loading
• File system writes outside allowed paths
• Network calls to non-whitelisted endpoints

This is not a complete defense — static analysis cannot catch everything. But it catches the obvious vectors before they reach execution. The goal is defense in depth: each layer catches what the previous layer missed.

Layer 2: Subprocess Isolation

Validated code executes in an isolated subprocess with restricted capabilities:

• No network access beyond explicitly whitelisted endpoints
• No file system access beyond a temporary working directory
• No environment variable access beyond injected credentials
• Resource limits on CPU time, memory, and wall clock duration

The subprocess cannot see other processes, cannot access the host file system, and cannot modify its own resource limits. If execution exceeds the timeout or memory cap, the process is killed and the task fails safely with a descriptive error.

Layer 3: Kubernetes Container Isolation

For Code Mode and Flow executions, the platform goes further with K8s-native container isolation. Each tool execution gets its own container with:

• Ephemeral pods that are created for the task and destroyed on completion
• Network policies that restrict egress to whitelisted endpoints
• Resource quotas enforced by the K8s scheduler
• No persistent storage — the container's filesystem is temporary
• Non-root execution with dropped capabilities

This level of isolation means a compromised tool execution cannot affect other workloads on the cluster, cannot access other namespaces, and cannot persist beyond the task duration.

Credential Injection in the Sandbox

Credentials enter the sandbox exclusively through environment variables, injected by the PlatformCredentialInjector at execution time. The tool code references environment variables like $API_KEY — it never knows the actual credential value until execution, and the credential is scoped and time-limited as described in our credential isolation post.

When the sandbox is torn down, environment variables are scrubbed and credentials are revoked. There is no window where credentials persist in a stopped container or a log file.

Risk-Based Execution Modes

Not every tool needs the same level of isolation. The platform assigns risk levels to tool executions and selects the appropriate sandbox tier:

• LOW risk (read-only, no side effects): Subprocess isolation with standard resource limits. Fast startup, minimal overhead.
• MEDIUM risk (writes data, modifiable state): Subprocess isolation with tighter resource limits and network restrictions.
• HIGH risk (deletes data, system changes): K8s container isolation with ephemeral pods and full network policies.
• CRITICAL risk (irreversible, security-sensitive): Requires human approval at the HITL gate before execution. K8s isolation with additional monitoring.

Why Not Just Use Docker?

Docker containers provide process isolation but not orchestration. For AI agent workloads where you need dynamic scaling, resource quotas, network policies, and automatic cleanup, Kubernetes is the right abstraction. The platform uses K8s to:

• Schedule tool execution pods on appropriate nodes (ARM64 vs AMD64, GPU vs CPU)
• Enforce cluster-wide resource limits so one runaway agent cannot starve others
• Apply network policies that restrict egress per-pod, not per-container
• Handle automatic cleanup when pods exceed their TTL

Our cluster runs K3s across mixed-architecture nodes — ARM64 Raspberry Pi workers for lightweight tasks and AMD64 nodes for compute-intensive work. SmartModelRouter and the scheduler work together to place tool executions on the right hardware.

The Full Stack

Sandboxed execution is one layer of the platform's security stack. It works with credential isolation (scoped tokens per invocation), HITL gates (human approval for risky operations), the Audit System (immutable logging of every execution), DLP scanning (sensitive data detection before model calls), and RBAC (role-based access to capabilities).

No single layer provides complete security. The architecture assumes each layer will have gaps. The combination of static validation, process isolation, container isolation, credential scoping, human oversight, and audit logging creates a defense-in-depth posture that is appropriate for enterprise environments.