Three open source layers to stop AI coding agents going rogue
Coding agents move faster than human review can keep up with. They read files, run scripts, and make network calls, all under your credentials. Sysdig’s research team built a demo combining three open source tools to show what a real defense-in-depth stack looks like when you hand Claude Code the keys to your repo.
️ The Three Watchmen
The stack uses NVIDIA OpenShell, Prempti, and Falco. Each answers a different question:
| Tool | Where it runs | What it answers |
|---|---|---|
| OpenShell | Sandbox boundary | What is the agent allowed to do? |
| Prempti | Inside the sandbox, as a Claude Code hook | What does the agent intend to do? |
| Falco | On the host, via eBPF | What actually happened? |
OpenShell creates a sandboxed container for the agent. A policy file specifies which filesystem paths are read-only versus read-write (enforced with Landlock) and which binaries can reach which network hosts. Everything outside the policy fails with EACCES. Allowed and denied network decisions are logged as OCSF events.
Prempti plugs into Claude Code as a PreToolUse hook. Before every tool call, Bash, Write, Edit, and others, Claude Code hands the call to Prempti, which runs it through Falco rules and returns allow, ask, or deny. It sees intent before execution.
Falco runs on the host outside the sandbox using eBPF. Nothing inside the sandbox can stop it, reconfigure it, or lie to it. Rules key on OpenShell container labels rather than container names, so they work for any sandbox regardless of image.
What the Five Test Scenarios Showed
The demo ran five scenarios ranging from direct credential access attempts to attacks on the watchmen themselves.
- Direct credential probe:
test -f ~/.aws/credentialsgets a Prempti deny immediately. The command never executes, so Falco sees nothing. Straightforward. - Indirect credential probe via Python script: Prempti sees two tool calls, write a Python file and run it, neither of which mentions credentials explicitly. Both Prempti and OpenShell allow it. Falco fires a Warning when
python3opens~/.aws/credentialsat the syscall level, even when the file does not exist yet (res=ENOENT). Indirection gets past intent-based tools. It does not get past the kernel. - Allowed egress is still visible: Node scripts the agent writes share the same egress permission as Claude Code itself. OpenShell logs the DNS mapping, the connection, and the HTTP request. Falco raises a Notice. Nothing bad happened, but you can tell Claude Code’s own API calls apart from scripts the agent wrote doing the same thing.
- Aggressive agent simulation: A test script tried to write into OpenShell’s runtime directory, delete the supervisor’s CA certificate, connect directly to
1.1.1.1, and kill process 1. All four failed at the sandbox boundary. Falco logged Critical and Warning alerts for every attempt because syscalls reach the kernel even when they fail. A blocked attempt is still a signal: an agent trying to tamper with its own sandbox is one you want to stop trusting immediately. - Killing Prempti: Prempti runs inside the sandbox as the same user as the agent. Its files are root-owned, so the agent cannot edit its rules directly, but it can send it a signal. On
SIGTERM, Prempti shuts down cleanly, removes its own hook from~/.claude/settings.json, and from that point forward nothing checks tool calls. One watchman down, failing open. The fix: move the hook to/etc/claude-code/managed-settings.json(root-owned, daemon cannot remove it) and configure Prempti to ask before any signal. With that change, killing Prempti requires human confirmation and causes the hook to deny everything instead of disappearing quietly.
The Coverage Table
| Action | OpenShell | Prempti | Falco |
|---|---|---|---|
test -f ~/.aws/credentials | never runs | deny | never runs |
| Python script reads credentials | allow | allow | Warning |
| Node script calls api.anthropic.com | allow, logged | not a tool call | Notice |
Write to /.openshell, connect to 1.1.1.1, kill pid 1 | deny | not a tool call | Critical / Warning |
| Kill Prempti’s daemon | allow (same user) | ask, then deny everything | Critical |
| Swap Prempti for a rule-less copy | allow | bypassed | Critical |
The Key Insight
No single layer was sufficient. Prempti catches obvious intent but cannot simulate script execution. OpenShell enforces hard limits but cannot distinguish the agent’s own API traffic from a script the agent wrote. Falco sees everything, but only after the fact and without context about what the policy permitted.
One practical finding from live sessions: Claude Code generates around 160 Falco alerts per normal session before any tuning, mostly blocked telemetry and connector calls. Three environment variables in the entrypoint and a few narrow rule exceptions brought that to zero. Tune against real agent sessions, not just scripted attacks.
Try It
The full demo is in the sysdiglabs/openshell-prempti-falco-demo repo. Requirements: Docker 28+, the Just command runner, and the OpenShell CLI and gateway. Tested on macOS Apple Silicon with colima, OpenShell 0.1.2, Falco 0.45, and Prempti 0.2.1.
just gateway # once: local OpenShell gateway
just falco # terminal 1: Falco with OpenShell rules
just openshell # terminal 2: Claude Code in a sandbox, with Prempti
just prempti-logs # terminal 3: Prempti's verdicts
No Anthropic API key? Run the simulated evil agent script directly and watch Falco fire without needing a live Claude session.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
