SandboxGym

Risk map

X = finding count · Y = escapes×100 + cross-tenant×10 + bypass

Critical High Moderate

SandboxGym

Last updated Next update

AI agents now browse, execute code, install dependencies, and reach private systems from inside "isolated" execution environments. We audited eight sandbox and virtual-machine products, from major labs to small startups, to measure how much of the advertised boundary actually holds.

Nearly every sandbox or virtual-machine implementation we examined contained a critical vulnerability capable of collapsing its intended isolation boundary, turning systems designed to safely contain untrusted code into potential entry points to the host, its data, and adjacent workloads.

See Results Nominate a sandbox/VM

Goal

SandboxGym exists so users can see the real security posture of the sandbox or VM they rely on for untrusted code — not only whether isolation fails, but how seriously vendors treat security and how responsibly they fix what we report.

Disclosure policy Each month we report findings to vendors and refresh this benchmark for the previous month. The public update ships on the 15th of every month.

Experiment setup

We selected eight open-source sandbox and virtual-machine projects for this audit. If an open-source sandbox belongs on this list and isn’t here yet, nominate it here.

Method

Each product was audited by Nebula Security’s code scanning agent against a fixed open-source snapshot. Scans covered the entire codebase at deep depth, with findings retained only when they were exploitable from inside the sandbox and could collapse the intended isolation boundary.

AuditorNebula Security
ScopeEntire codebase, deep scan depth
SnapshotFixed open-source commits, see appendix
8
Products surveyed
36
High-impact isolation failures, all exploitable from inside the sandbox
5
Full escapes / host code execution & memory corruption
100%
Of scored products had at least one isolation failure
01

Finding count ranking

Ranked by total finding count, highest first. Every product we scored had at least one isolation failure; the breakdown distinguishes full escapes, policy bypasses, and cross-tenant access.

More findings do not necessarily mean higher risk; failure class and impact still matter.
Full escape / host memory corruption Policy bypass Data exposure / cross-tenant

Finding composition by product

Stacked by failure class and ordered by total finding count.

Product Isolation model Escape Bypass Cross-tenant Total Disclosure

Counts include only issues with direct host memory corruption, host file overwrite, network-policy bypass, or impact on other sandboxes on the host. Unauthorized out-of-sandbox file reads and GPU-dependent findings are excluded from these totals.

Commit activity

Each point is a commit from Feb 9 through July 15, 2026; denser bands mean higher update frequency. Red markers are introducing commits for attributed findings (probable-cause analysis, not raw HEAD blame).

Commit activity shows how often developers maintain and update a product — how quickly they ship fixes, and how often they introduce new issues. That cadence helps assess overall security posture and engineering maturity.

smolvm tends to introduce issues in a short window — dense red markers during intensive feature work, when security review does not keep pace with shipping. They did fix vulnerabilities once we reported them, which shows they take disclosed issues seriously, but lack an effective way to catch them during development.

runtm has the lowest update frequency for both bug fixes and new features. The vulnerabilities we reported were never fixed.

beta9 began intensive feature work in late May. Development after that is steady, and relatively few vulnerabilities were introduced in that burst compared with the volume of commits.

cua often lands large changes as monolithic PRs. Every counted vulnerability was introduced in a single commit (7d1fa31: ~30k lines added across 197 files) — again pointing to missing security gates during development rather than a slow drip of regressions.

sandbox-runtime’s published criticals are a late-June Windows ACL cluster: the separate-user / deny-ACE foundation land in 4785dcc (2026-06-26), and the follow-on refactor that drops group/ACL-stamp modes in daef9ed (2026-06-30) introduces the remaining criticals. Red markers on the sandbox-runtime lane are snapshot-blame introducing commits for those findings.

nono is the fastest on remediation: they acknowledged and fixed reported issues in about two days — the quickest response in this set. Their developers also noted they are still in fast development, so vulnerabilities are expected for now; we expect security focus to increase as the product stabilizes.

amika provides a Docker-based runtime. Relying on mature Docker isolation means severe runtime escapes are rare, but it also means they have little control over the sandbox engine itself — risk shifts to mount semantics, control-plane APIs, and configuration mistakes around Docker rather than a custom isolation boundary.

02

Disclosure posture

A working disclosure channel is part of a product's security posture. For products whose core purpose is executing hostile code, a monitored SECURITY.md, security address, or bounty channel should be a baseline requirement. We thank nono and smolvm for prompt, detailed responses.

03

Where the boundary breaks

Grouped into the three failure classes we track: full escapes, policy bypasses, and data exposure / cross-tenant access.

Full escape / host memory corruption 514%
Policy bypass 2878%
Data exposure / cross-tenant 38%
Dominant failure mode The most common weakness was policy bypass. Building a sandbox or VM directly from low-level primitives gives developers more control, but transfers far more security responsibility to the implementation, path resolution, mounts, inherited handles, privileges, routing, and cleanup all become part of the boundary.
Count ≠ exploitability More findings does not automatically mean less secure. smolvm has the most findings (10), but it drops privileges after boot, so reaching arbitrary host execution requires chaining multiple bugs, a meaningfully harder and more secure posture than a single one-line path to host command execution.
04

Case Study

The most representative failure patterns, from a sandbox that never engages, to a guest that rewrites host memory.

runtm · RuntimeFull escape · host-equivalent

Sandbox prompts execute directly on the host

TLDR, runtm session prompt runs outside the sandbox entirely, even though it claims to run via run_prompt_in_sandbox.

The shipped prompt path never launches the sandbox-runtime isolation layer. It forwards the prompt to the Claude adapter, which starts the host claude binary directly with --dangerously-skip-permissions and only cwd=workspace. Because the filesystem and network guardrails are srt settings enforced only when a process is launched through srt, agent actions run with the same OS permissions as the runtm process, not the intended sandbox restrictions.

advertised: run_prompt_in_sandbox→ CLI path skips srt→ host claude --dangerously-skip-permissions→ full host permissions
smolvm · Smol MachineFull escape · host ACE

DAX mapping offsets wrap and remap arbitrary host pages

TLDR, an in-sandbox attacker can re-map memory in the VMM and gain arbitrary code execution in the smolvm process.

The virtio-fs DAX path trusts guest-supplied shared-memory offsets. On FUSE_SETUPMAPPING / FUSE_REMOVEMAPPING, the host reads raw moffset / len and only checks (moffset + len) > shm_size. That addition is not overflow-safe before deriving a host address and calling mmap(..., MAP_FIXED, ...), so a wrapped offset targets memory outside the virtio-fs window.

guest FUSE mapping→ offset + len wraps→ mmap MAP_FIXED out of window→ VMM code execution
sandbox-runtime · AnthropicPolicy bypass

Inconsistent Windows sandbox enforcement

TLDR, filesystem policy is applied inconsistently across Windows execution paths.

The Windows backend does not enforce filesystem policies consistently across all supported execution paths. Restrictions that appear enabled at the configuration layer may be partially applied, interpreted differently, or left unenforced by the underlying Windows sandbox mechanism, a gap between the advertised policy and the boundary actually enforced.

policy enabled in config→ path-dependent enforcement→ restriction silently not applied
amika · AmikaPolicy bypass · persistent host write

rwcopy mounts become writable host bind mounts

TLDR, a malicious process in the sandbox can persistently modify the original host file.

The sandbox-creation API accepts mount entries with Mode: rwcopy, but the shared service path never turns the request into an isolated copy. It passes the mount straight to Docker, so every rwcopy bind mount writes persistently to the original host files.

mount Mode=rwcopy→ no copy-on-write→ raw Docker bind mount→ persistent host file write
hypeman · KERNELPath traversal · unauthorized deletion

Snapshot ID path traversal escapes the snapshot root

TLDR, a path traversal in snapshot handling leads to unauthorized file deletion outside the store.

Snapshot deletion and lookup accept a caller-supplied snapshotId and use it as a path component without confinement. Passing .. into filepath.Join resolves to the parent of the snapshot store, and os.RemoveAll then deletes that directory tree, an unauthorized filesystem operation outside the snapshot sandbox root.

caller-supplied snapshotId→ ".." via filepath.Join→ resolves above store root→ os.RemoveAll deletes tree
05

Takeaway

Using a container, virtual machine, or sandbox does not automatically make an execution platform secure. Across the products we examined, the isolation boundary was repeatedly weakened, even broken, by the surrounding implementation: unsafe mounts, inconsistent policy enforcement, path-handling mistakes, privileged helpers, shared state, and management APIs. Findings such as GhostLock further show an attacker can reach kernel-level privilege from a heavily constrained user. Isolation should not be treated as a vague product claim; vendors should clearly document, disclose, and remediate failures that materially weaken the promised boundary.

smolvm

Smol Machine
Stack Rust microVM on libkrun (KVM / HVF / WHP) Findings 4 escape · 6 bypass Response Responded · 3 days

Smolvm sells a real guest kernel per workload, stronger than shared-kernel containers in principle. The high-impact issues we counted sit on the VMM and packaging surfaces around that boundary, for example: guest-trusted virtio-fs DAX mapping arithmetic that remaps host pages, writable shared rootfs / packed layers. More findings here does not equal weakest product: privilege drop after boot still raises the bar for host-equivalent impact relative to sandboxes that never engage.

Disclosure was among the strongest in the set (reply in three days; no SECURITY.md, but other channels worked). 5 out of the 10 vulnerabilities have been fixed. Vulnerabilities triggerable in sandbox or when loading the payload are in scope.

runtm

Runtime
Stack OS-policy sandbox via Anthropic sandbox-runtime (srt) Findings 1 escape · 1 bypass Response No reply yet

Runtm advertises OS-level isolation by wrapping agent work in srt (bubblewrap / seatbelt-style policy). The critical finding is architectural: the shipped session prompt path never launches that layer and instead runs host claude --dangerously-skip-permissions, so the advertised boundary is absent on the primary UX path. A second issue lets writable on-disk sandbox state retarget later prompts outside the assigned workspace.

Security contact posture was poor (SECURITY.md email undeliverable; no reply at publication). We reached out to the vendor for confirmation through their other public email addresses in July, but did not receive a response by publication.

beta9

Beam
Stack Container sandbox runtime (Beam documents gVisor + runc) Findings 0 escape · 7 bypass · 1 cross-tenant Response No reply yet

Beta9 is Beam’s open-source sandbox engine: orchestration, volumes, and code execution for AI workloads. Public docs emphasize container isolation with gVisor. The counted failures are mostly control-plane and filesystem mediation bugs, workspace sync traversal, cachefs percent-encoding materializing daemon-local files, source-fetch reading attacker-chosen host paths, and symlink following onto the host. That pattern weakens the container story even when the guest runtime itself is sound.

SECURITY.md exists; we had no reply by publication. We can not confirm whether the cloud version is affected by these issues as we did not receive a response from the vendor.

hypeman

KERNEL
Stack Multi-hypervisor OCI VM runtime (Firecracker, Cloud Hypervisor, QEMU, Apple VZ) Findings 0 escape · 5 bypass · 1 cross-tenant Response No reply yet

Hypeman puts OCI images into real VMs behind a Docker-like CLI, a strong isolation model when the guest is the only trust boundary. Findings clustered on host-side orchestration: snapshot-ID path traversal into os.RemoveAll, volume / size cleanup deleting arbitrary host directories, unrestricted host PCI function registration, rootfs symlink redirect of pre-chroot secret writes, and a Firecracker standby-fork path that retains a live pointer into source VM snapshot memory. The hypervisor can be fine while the management plane still breaks the store.

No SECURITY.md; no reply yet. We reached out to the vendor for confirmation through their public email address, but did not receive a response by publication.

cua

Cua
Stack Computer-use agent infra, Lume / Docker / QEMU / Hyper-V backends Findings 0 escape · 3 bypass · 1 cross-tenant Response No reply yet

Cua’s job is isolated desktops for computer-use agents, with multiple providers rather than a single sandbox primitive. Issues we counted include host command execution via PowerShell metacharacters in Hyper-V parameters, sandbox-name and layer-title path traversal writing host files, QCOW2 pull artifacts that can make QEMU open arbitrary host paths, and every Hyper-V guest exposing an unauthenticated administrator command server on the host. Guest OS isolation does not help if the control path or guest agent is wide open.

No SECURITY.md; no reply yet. We reached out to the vendor for confirmation through their public email address, but did not receive a response by publication.

sandbox-runtime

Anthropic
Stack OS-policy sandbox (bubblewrap / seatbelt / Windows ACL + network proxy) Findings 0 escape · 3 bypass Response Not on HackerOne yet

Anthropic’s srt is a lightweight process sandbox used by Claude Code and by dependents such as runtm: filesystem and network policy without a container or VM. The product is the policy engine, so inconsistent enforcement is the core risk. Representative counted issues include Windows filesystem policy applied inconsistently across execution paths; additional Windows ACL / hardlink / proxy-policy issues appear in the scan corpus but introducing-commit attribution for this product remains deferred.

Anthropic is on HackerOne generally; this experimental repo was not yet listed there at review time.

nono

nolab
Stack Kernel OS-policy (Landlock, Seatbelt, seccomp notify, network proxy) Findings 0 escape · 2 bypass Response Responded · 2 days

Nono applies allow-lists to agent processes. That design targets the right threat model for local agents. Counted failures are enforcement gaps rather than full escapes: seccomp network approvals continuing the original syscall after authorizing a child, and numeric /dev/fd reopen into the supervisor’s descriptor table. Small finding count plus fast, detailed reply put nono among the better postures in this set.

SECURITY.md present; replied in two days. Nono is still under experimental and rapidly development. They have realized the issues and are working on fixes.

amika

Amika / Fixpoint
Stack Docker sandboxes today; docs also describe microVM / multi-provider control plane Findings 0 escape · 1 bypass Response No reply yet

Amika is a control plane for coding-agent sandboxes: preset Docker environments, credential mounts, and an HTTP API. The primary counted issue is semantic: Mode: rwcopy mounts are documented as copy-isolated but become live writable host bind mounts, so in-sandbox writes persist on the host. Related scan material also showed a materialize path executing request-supplied commands on the host rather than inside a container.

No SECURITY.md; no reply yet. We reached out to the vendor for confirmation through their public email address to verify the expected behavior, but did not receive a response by publication.

06

Appendix · audited commits

Every finding was reproduced against a fixed open-source snapshot. Results may not apply to later releases, private deployments, or differing configurations.

Severity ratings describe potential impact when successfully exploited, not exploitation complexity. The inclusion of a company or product does not imply that every deployment is vulnerable, nor does the absence of a product imply that it is secure. This survey is a snapshot of the implementations reviewed, not a complete evaluation of the isolated-execution market. Several GPU-related and out-of-sandbox read-only findings were excluded from the primary statistics.