Homelab Rebuild Part 38: An Autonomous AI Pentester in a Sealed Box

| October 23, 2026 | 11 min to read

Intro

I wanted to point an autonomous AI pentester at my own public demo, homie-demo.franzramadhan.dev, and let it probe the app the way an external attacker does. The tool is PenTAGI (vxcontrol/pentagi). You give it a target and a goal, and a crew of LLM agents plans the attack, runs tools and reads the output. The catch is that you do not want an autonomous attacker anywhere near your own infrastructure. So before I asked what it can find, I had to answer a different question. Can I run it without any path into my homelab cluster?

TL;DR: run it inside a throwaway VM that is network-isolated at the VM boundary. It stays fully contained even when it spawns its own attacker containers, because the containment lives below anything the agent can touch. But on Cloudflare Workers AI, the agent is too weak at tool-calling to drive a pentest. It made 47+ LLM calls, and it produced zero tasks, zero tool runs and zero spawned containers.

The Figure

The agent being held: PenTAGI, root, Docker-socket-nativeCapped conduits: the LAN and the cluster, dropped at the VM boundaryTwo thin lines out: the LLM proxy and the public targetThe chamber is a throwaway VM. Inside sits the autonomous AI pentester (PenTAGI), with the attacker containers it would spawn beside it. Containment is enforced at the VM boundary, so it holds whatever the agent or a spawned container decides to do. The left conduit (the LAN, including the gateway) and the back conduit (the k0s cluster) are capped by the Proxmox firewall group. Two lines leave the east wall: one scoped tailnet grant to the LiteLLM proxy on 443, and general outbound internet to the public demo through a Cloudflare edge. Switch the probe between the VM and a container it spawned: both see the LAN and the cluster blocked and the target reachable. On Workers AI the agent stalls, 47+ LLM calls and zero tasks, tool runs or spawned containers.

The figure is a sealed cell on a bench. Inside sits the agent (the tall lit server), with the attacker containers it can spawn beside it. Two routes out of the cell are capped bulkheads with a crossed-out plate: the LAN (and its gateway) and the k0s cluster. Two thin lines leave the other side. One goes to the LiteLLM proxy box. The other goes through a cloud edge to a globe, the public target. Four lamps on the front wall read LAN blocked, cluster blocked, LLM allowed, target allowed. Click the agent, a bulkhead or the target to list the probes that pass. Press V to run the probe from the VM or from a container the agent spawned. Both see the LAN and the cluster blocked and the target reachable. I built it with the anatomy skill.

The Situation

The obvious place for a new workload in my homelab is the k0s cluster. PenTAGI is the wrong workload for it. The reasons all come down to how it runs.

PenTAGI is Docker-socket-native. It mounts the Docker socket, runs as root, and spawns a fresh container for each pentest “flow” (one run against a target). That gives the agent a clean box to work from. The design assumes a Docker daemon it can drive.
My cluster has no Docker socket, by design. It is k0s on top of containerd, so there is no /var/run/docker.sock to mount. Also, the cluster is locked down. Kyverno denies privileged pods, and Tetragon kills a process on a policy hit. A privileged docker-in-docker pod fights every one of those guardrails at once. I built those guardrails on purpose (Part 17 and Part 34).
A microVM is too small. My microVM sandboxes (Part 23) are tiny on purpose. PenTAGI needs a full Docker daemon that itself spawns Kali containers. It also needs a pgvector database for the memory of the agents and a scraper that asks for a 2 GB shared-memory segment. That does not fit.

So the answer was a dedicated full VM: one Debian 13 guest, off by default, started by hand only for a run. The isolation lives at the network boundary of the VM, not inside the tool or inside Docker. So it sits below anything the agent (or a container the agent spawns) can reach. This is the same lesson as my guest-VM isolation post (Part 33): put the wall where the tenant cannot get under it.

Setup

The VM

I defined the VM in Terraform. It is a clone of my Debian 13 template, with 4 vCPU, 8 GiB of RAM and a 40 GB disk. I kept it out of the Terraform group that defines my k0s nodes, so it does not inherit their autostart. It carries a no-autostart tag, and it does not start on host boot. The nightly power schedule that stops and starts the cluster leaves it alone. It stays stopped, and I start it by hand when I want a run.

Isolation layer 1: the LAN

The first wall is the Proxmox firewall. The NIC of the VM has the firewall enabled and points at a firewall group that:

drops traffic to the home subnets, including the gateway,
allows the router only on port 53 so DNS still works,
leaves general outbound internet open.

The result is a VM that can reach the internet and resolve names, but cannot touch anything else on the LAN. My isolated guest VM uses the same firewall group, so it is a pattern I already trust.

Isolation layer 2: the tailnet

The VM also joins my tailnet, so I can reach its web UI without exposing it. The risk with the tailnet is the opposite of the LAN. By default, a device on it can be quite reachable. So the VM gets a dedicated tag with exactly one scoped grant (a Tailscale access rule). It lets the VM reach only the LiteLLM node on port 443 and nothing else:

// one grant, nothing else matches this tag
src:  tag:pentagi
dst:  the LiteLLM node
ip:   tcp:443

This is airtight because the LiteLLM node is itself a tagged device, not one owned by my user. So the broad “allow my own devices to reach everything” grant does not match it. The VM joins the tailnet already tagged (tailscale up --advertise-tags=tag:pentagi). My tailnet has device approval turned on. So the machine sat in a “needs approval” state until I approved it in the admin console. Only then did tailscale up return success.

LLM via LiteLLM

The agents talk to my in-cluster LiteLLM proxy (Part 29), which is Cloudflare Workers AI only. That makes this experiment-grade from the start. Workers AI is cheap and private, but it is not a frontier model. PenTAGI also wants embeddings for its pgvector memory. So I added a bge embedding model to the proxy and made sure that it returns the right number of dimensions.

The custom OpenAI-compatible provider in PenTAGI points at the proxy. All 13 agent roles map to the smart complexity router of the proxy. One line cost me an afternoon:

# PenTAGI .env: for a LiteLLM proxy whose model names are bare, leave this EMPTY
LLM_SERVER_URL=https://litellm.<tailnet>/v1
LLM_SERVER_MODEL=smart
LLM_SERVER_PROVIDER=

LLM_SERVER_PROVIDER is a prefix that PenTAGI prepends to the model name for provider routing. My model names are bare (smart, not openai/smart). If the variable is set, PenTAGI sends prefix/smart, and the proxy answers HTTP 400. An empty value fixes it.

The stack

A single docker compose up -d brings up PenTAGI plus its pgvector database, a scraper, and a Postgres exporter. The migrations run, and the API comes up on TLS. The UI is reachable over TLS on the LAN, and the default login is admin@pentagi.com / admin. You must change that on first login. It is a console that drives an autonomous attacker. It is the last thing you want on a default password.

What Happened When I Ran It

Containment first

Before I pointed it at anything, I checked the walls. From the VM itself, the allowed paths work and nothing else does. Then I had PenTAGI spawn a container (the same mechanism it uses for a real flow). I ran the same probes from inside that container, because that is where the agent works. The container could reach the public target. It got nothing to the LAN gateway or the k0s controllers, because the firewall blocked the connection.

Destination	From the VM	From a spawned container
the LiteLLM proxy on 443	open	not on this path
DNS via the router	open	not tested
general internet	open	open
`homie-demo` (the target)	HTTP 200	HTTP 200, via a Cloudflare edge IP
the LAN gateway	blocked	blocked
a k0s controller or worker	blocked	blocked
any other tailnet host	blocked	not reachable

The last column is the one I cared about. The wall also covers the containers that PenTAGI spawns. They sit inside the VM, and the wall is at the VM boundary. No rule inside PenTAGI or Docker is involved. The wall holds no matter what the agent decides to do, because the agent never gets to decide anything below that line. The request to the target went out over the internet and came back through a Cloudflare edge IP. That also shows that the “attack” really leaves the house and comes back through the front door. It does not find an internal shortcut.

The agent itself

With containment proven, I started two flows with a tightly scoped prompt, restricted to only homie-demo. Then I watched it do almost nothing.

The agent made 47+ LLM calls (about $0.15 on Workers AI). It produced zero tasks and zero tool executions, and it spawned zero sandbox containers. It stalled in planning. The PenTAGI planner needs the model to return structured tool-calls that it can execute. Workers AI does not do that reliably enough to get past the first step. The crew talked to itself and never picked up a tool.

One thing nearly fooled me while I watched the traffic. During the run, the only non-LLM outbound connections from the VM were the Tailscale DERP relay connections (Tailscale relay servers), owned by the tailscaled process. At a glance, it looks like the agent reaches out to something. It did not. It was my own tailnet that kept its relays warm.

Lessons

Put containment at the VM boundary, and prove it from inside a spawned container. A wall the tenant cannot get under beats any number of rules that the runtime of the tenant is supposed to enforce. Test it from where the agent runs, not only from the host.
A locked-down Kubernetes cluster is the wrong home for a Docker-socket-native tool. Kyverno and Tetragon exist to stop exactly the kind of privileged, root, socket-mounting workload that PenTAGI is. A throwaway VM is the right home, because there is nothing there to protect.
Cloudflare Workers AI is too weak at tool-calling to drive an agentic pentester. The containment works, but the pentest does not. You need a capable function-calling model behind the proxy before the agent picks up a single tool.
Watch egress to check scope, and know your own background noise. The only traffic that left the VM during the run was Tailscale DERP relays. If you do not recognise the chatter of your own infrastructure, you will mistake it for the tool.

Series

This post is part of Homelab Rebuild: