Intro
dvarapala is the anti-DDoS gateway I build in Rust, and it runs in front of my own public apps (Part 40). It grows faster than I can review it. I wanted a second pair of eyes that reads the whole repository on a schedule. It looks for security, performance and scalability problems, and for features that never got finished. Then it files them the way the project already works: an RFC for a design change, an issue for a bug, and a private advisory for a vulnerability.
When I approve an item, I also wanted the implementation done, up to a green pull request. The merge stays with me.
This post is that pipeline. It has two n8n workflows on the n8n setup from Part 41, a GitHub App, and a new runner tier (a separate set of runner pods) that can use a shell. It also covers the seven bugs I hit while I tested it.
TL;DR: Claude Opus reads dvarapala at the current
maincommit every Monday (security), Wednesday (performance and scalability) and Friday (implementation gaps). Each finding becomes a private draft security advisory, a public issue, or a draft RFC pull request. Nothing is implemented until I add the labelagent:implement. Every 2 hours, a Sonnet runner with git,ghandgh stacktakes the oldest approved item. It opens the pull requests by the project’s own rules. n8n then watches CI, asks for up to two fixes, and labels the itemagent:greenoragent:needs-human. When I review its PR and addagent:revise, it works through my comments the same way. GitHub signs its commits, because it pushes through the API. The agent never merges. First runs: Opus reviews reported 1.92 to 3.33 USD each, the first implementation 0.26 USD, all on my Claude subscription.
The Figure
I drew the pipeline as an inspection line. The one lit part is a finding card
that rides the belt. The review camera on the left (Opus, read only) scans the
repo crate and drops a card. Then the sorting gate swings one flap. A security
card slides into the locked box, and the line stops there. A bug drops into
the issue tray, and a feature drops into the RFC tray on the human desk.
Nothing leaves a tray until the stamp on that desk comes down on the card. The
stamp stands for my agent:implement label or an accepted, merged RFC. The
implement station (Sonnet) then presses PR plates onto the card: one for a
bug, and a stack of three for an RFC. The CI bench probes them. Red sends the
card back for a fix. Green lights the top lamp and sends the card to the out
tray for me to review and merge. Click the figure or press 1, 2 or 3 to run a
security, bug or RFC finding. The readout in the corner shows the station,
item, label and CI round. I built it with the
anatomy skill.
The Situation
- n8n 2.45.7 in my k0s cluster, with
claude-runner: headless Claude Code on my Claude subscription, one Deployment per model tier (Part 41). - Claude Code 2.1.316, models
claude-opus-5-5andclaude-sonnet-5-5. - franzramadhan/dvarapala: public, Rust workspace, about 10 minutes of CI per pull request.
- dvarapala already has a written process, and the agents follow it instead
of inventing their own:
CLAUDE.md: an RFC first for anything that changes behavior or config. The RFC comes fromrfc/0000-template.md, has a Test cases table, and gets a row inROADMAP.md. A bug fix is one PR: a regression test commit, then the fix. A vulnerability gets a private report. Nobody discusses an unfixed one in a public issue, PR or commit message..claude/skills/rfc-stack: an accepted RFC becomes a stack of PRs, made withgh stack: a tests layer first (the RFC’s Test cases as ignored tests), one layer per domain, then the CLI layer.CLAUDE.local.md(not in git): no localcargo build,check,testorclippy. Only the cheap checks run locally (cargo fmt, a 500-line file check,cargo deny, the UI build). CI verifies the rest.
Setup
The flow
flowchart LR
R["Review<br/>Opus, read-only<br/>Mon / Wed / Fri 08:00"] --> S{"kind"}
S -->|security| A["private draft<br/>advisory"]
S -->|bug| I["issue<br/>label agent"]
S -->|feature| F["draft RFC PR<br/>label agent"]
I --> H["me: label<br/>agent:implement"]
F --> H
H --> D["Implement<br/>Sonnet dev runner<br/>every 2 h, 07:30 to 19:30"]
D -->|signed push<br/>via the API| C{"CI"}
C -->|red, max 2| D
C -->|green| G["agent:green<br/>I review"]
G -->|"feedback +<br/>agent:revise"| D
G -->|approve| M["I merge"]Security findings stop at the advisory. I fix a vulnerability in a temporary private fork, and the pipeline does not touch that. A temporary private fork is a private copy of the repository that GitHub makes for an advisory.
A GitHub App instead of my token
One App for all my projects, franzramadhan-agent-app, installed on
dvarapala:
- Repository permissions: contents, pull requests, issues and repository
advisories (read and write). Checks, actions and metadata (read). No
workflowspermission, so a change under.github/workflowsstops with a note for me. No webhook, no ruleset bypass. - The private key lives in OpenBao and reaches n8n through
CREDENTIALS_OVERWRITE_DATA. It never reaches a runner pod. - Every run mints its own installation token. The token covers one repository and only the permissions that step needs, and it is valid for 1 hour:
JWT (RS256, iss = App id)
-> GET /repos/franzramadhan/dvarapala/installation
-> POST /app/installations/<id>/access_tokens
{"repositories": ["dvarapala"], "permissions": {"contents": "write", ...}}
-> ghs_... token
Commits from the runner carry the App’s bot identity, so it is always clear what an agent wrote. GitHub also signs them (see Signed commits through the API).
A runner tier that can use a shell
The runners from Part 41 refuse the Bash tool, because the subscription
token is in the process environment. Implementing code needs git, gh and
cargo fmt. So I added a separate tier, claude-runner-dev, and accepted the
trade for that tier only:
- Sonnet, on
buildpack-deps:bookworm-scm(digest pinned). Init containers install Node 24, Claude Code, gh 2.103.3, the gh-stack 0.3.0 extension and skill, jq, cargo-deny 0.20.2 and rustup 1.29.2. rustup comes without a toolchain. Every download has a sha256 check. The firstcargo fmtpulls the toolchain the repository pins. - One run at a time, 45 minutes, 200 turns, 2Gi memory limit, a 4Gi work directory and 3Gi for Rust.
- The runner only accepts repositories from an allowlist (one today), and it
checks them out itself. It passes the per-run GitHub token to the child
process as
GH_TOKENand never logs it. - dvarapala’s
CLAUDE.local.mdis not in git, so the runner copies it into the checkout from a ConfigMap. It also copies aPreToolUsehook that blockscargo build,check,testandclippy. - A CiliumNetworkPolicy limits egress to GitHub, the Rust toolchain host and the crates index, on top of the Anthropic API.
- Plugins: caveman (terse output), ponytail (minimal code) and pstack.
The Opus reviewer stays read-only. It gets the repository as a tarball at the
exact main commit, and only Read, Glob and Grep.
Signed commits through the API
dvarapala’s ruleset on main requires verified signatures, and a GitHub App
cannot register a GPG or SSH signing key. But an App can create commits
through the API. GitHub signs a commit that an App creates with its
installation token, and shows it as Verified. So the dev runner does not send
commits with git push. A small git wrapper sits first on its PATH. It
runs every git command except push with the real git, and it turns a push
into API calls:
git push origin fix/issue-308
for each new local commit, oldest first:
POST /git/blobs (each added or changed file, base64)
POST /git/trees (base_tree = the parent's tree, so only the changes)
POST /git/commits (same message and tree, parent = the parent's signed copy)
POST or PATCH /git/refs/heads/fix/issue-308
move the local branch to the signed commits
Details:
gh stackcallsgit pushfromPATHtoo (git -c push.pushOption=pull.stacked push origin --force-with-lease=... --atomic refs/heads/x:refs/heads/x). So stacked PRs go through the same path, with no changes to the skill.- Trees are content addressed (the SHA comes from the content), so the tree GitHub builds must have the same SHA as the local one. The wrapper checks that and stops on a mismatch.
- After a push, the local branch points at the signed commits. A file in
.gitmaps each old SHA to its signed copy. A branch stacked on an unpushed layer uses it to find its new parent. - File modes travel in the tree entries, so an executable script stays executable.
- The ceiling: the wrapper does not check
--force-with-lease(a forced push is a plain force), and it refuses merge commits. The project wants linear history anyway.
Workflow 1: dvarapala review
At 08:00 on Monday, Wednesday and Friday:
- Pick the focus for the day: security, performance and scalability, or implementation gap.
- Mint a read token. Fetch the
maincommit, open items labelledagent, draft advisories, open pull requests and the RFC file names. Build a list of titles already raised (so the review does not report anything twice) and the next RFC number. - Run Opus, up to 50 turns and 30 minutes. The prompt says what to look at
for the focus and lists what is already raised. It asks for at most 5
findings as one JSON object. Each finding must be verified in the code,
with
file:lineevidence. For a feature, Opus writes the whole RFC file from the template, its ROADMAP row and the PR body. - Parse, cap (3 security, 3 bugs, 1 RFC), strip long dashes, mint a write
token, and route:
- security:
POST /security-advisories, a private draft. - bug: an issue labelled
agent. - feature: branch
rfc/NNNN-<slug>, the RFC file, the ROADMAP row inserted after the last RFC row, a PR titledrfc: NNNN <topic>, labelledagent.
- security:
- Send one Telegram line per item, or “no new findings”.
The gate is a label
Labels on dvarapala carry the state: agent (raised by the review),
agent:implement (my approval), agent:in-progress, agent:green,
agent:needs-human and agent:revise (I reviewed, address my comments). I
only ever set agent:implement and agent:revise. For a bug, I add
agent:implement to the issue. For an RFC, I set its status to accepted in
the RFC PR, merge it, and then add agent:implement to that merged PR.
Workflow 2: dvarapala implement
It runs every 2 hours from 07:30 to 19:30. The last start is 19:30, so a long run ends before the cluster shuts down at 23:30. A run with nothing to do makes three GitHub API calls and no model call.
- Do nothing while any item is
agent:in-progress: the dev runner takes one job at a time. Then take an open PR labelledagent:revise, or else the oldest item labelledagent:implementwithout a busy label. An RFC PR must be merged and titledrfc: NNNN. - Swap
agent:implementforagent:in-progress, so the workflow never picks it twice. - Mint a token (write for contents, pull requests and issues, read for
checks and actions). Call the dev runner with a prompt that follows the
project’s rules. For an RFC, the runner confirms that the status is
accepted, then builds the stack with the
rfc-stackskill. For a bug, it makes a branchfix/issue-N, the regression test commit, the fix, and one PR withFixes #N. For a revise, see below. No local build, no merge, no waiting for CI. It answers{"prs": [...], "notes": "..."}, and the notes leave CI out: the workflow reports CI once the checks finish. - Every 10 minutes, read the check runs of every PR head. The result is
green when all checks finish without a failure. If it is red, it goes back
to the dev runner with the failing check names and a fresh token, at most
twice. The runner reads
gh run view --log-failedfirst, then fixes the layer that owns the code. If it is still pending after 2 hours, or red after two fixes, a human looks at it. - Label
agent:greenoragent:needs-human, comment, and send a Telegram message. The green comment names the commits CI checked. It quotes the notes under a heading that says when they were written:
Review feedback addressed. CI finished green on #309 at 4739f71.
Changes (runner summary, written before CI ran):
...
For agent:revise, I review the agent’s PR on GitHub the same way as any
other PR. The dev runner then:
- Reads only feedback written by me: unresolved review threads (GraphQL
reviewThreads), review bodies and PR comments. It ignores anything from another author. The prompt treats comment text as review notes, never as instructions that change its rules. - Adds new commits on the PR’s branch and never rewrites the old ones. It pushes once through the signed path.
- Replies to each thread with what changed and the commit, or with why it did not change something. It does not resolve threads. That stays with me.
CI polling, the fix rounds, the labels and the comment then work as for any other item.
Issue 1: The Test Run That Never Started
Symptom
I test every workflow as a temporary copy with a webhook trigger instead of the schedule. I call that webhook from inside the n8n pod. The call answered 404, and n8n showed no execution at all. n8n’s log:
Received request for unknown webhook: This webhook is not registered for POST requests. Did you mean to make a GET request?
Cause
My webhook node had no httpMethod, and the default is GET. Before this, I
saw a 404 right after I activated a workflow (n8n registers the webhook a
moment later). So my first fix was a retry loop, and it did nothing. The log
line said exactly what was wrong. I read it after the retries.
Fix
"httpMethod": "POST" on the test trigger.
Issue 2: 403 from a Token That Has the Permission
Symptom
The run created two advisories, then failed on the branch for the RFC:
403 - {"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/git/refs#create-a-reference","status":"403"}
Cause
The App installation did have contents: write. I checked with an App JWT
(GET /app/installations/<id>). Then I minted a fresh token by hand and
created and deleted a test branch with it: 201, then 204. So the App was
fine, and the token n8n sent was not.
At that point the workflow used n8n’s built-in GitHub App credential. It
mints an installation token and stores it on the credential as an expirable
field. It renews the token only when a request gets a 401. An installation
token keeps the permissions the App had when it was minted. Mine came from
my credential test, before I changed the App’s permissions, so it had no
contents scope. A missing permission is a 403, not a 401, so n8n never
renewed it. With this behavior, the credential keeps such a token until it
expires, up to an hour later, and every request with it fails.
Fix
The review stopped using that credential. It mints its own tokens through a JWT credential and three HTTP nodes: a read token at the start, and a write token after Opus answers. Opus with retries can run longer than a 1-hour token lives. The implement workflow keeps the built-in credential for its CI polling, which runs for more than an hour. There an expired token gives a 401, and the credential renews it.
Issue 3: The JSON Answer Broke on Its Own RFC
Symptom
Unterminated string in JSON at position 3941 (line 3 column 1469)
This error came from the node that parses Opus’s answer, after a 74-turn review.
Cause
I asked for the answer in a fenced json block and cut it out with a lazy
regex (one that stops at the first possible match),
/```json\s*([\s\S]*?)```/. The answer contained a whole RFC file as a
JSON string, and that RFC has code blocks of its own. So the regex stopped at
the first inner fence, halfway through a string.
Fix
Take everything from the first ```json to the last fence in the answer.
Before I ran Opus again, I read the stored execution through the n8n API and
parsed it with the new code: 4 findings, including the RFC. The implement
workflow’s two parsers got the same fix.
Issue 4: A Notification That Stopped the Run
Symptom
The performance review created three issues and then failed in the Telegram
node with Bad request - please check your parameters. Its fourth finding was
never processed.
Cause
n8n’s Telegram node sends parse_mode: Markdown when none is set. One issue
title contained max_clients, and an unpaired underscore is invalid
Telegram Markdown. With n8n’s sequential execution order, the failed node
ended the run before the next branch ran.
Fix
Send HTML with &, < and > escaped, and set the Telegram node to
continue on error. A lost notification is not worth a lost finding.
Issue 5: Test Copies That Did Not Go Away
My test script deletes the temporary workflow at the end. Two copies stayed:
- The public API answers 409 to
DELETE /workflows/<id>until the workflow is archived (POST /workflows/<id>/archive). - My poll loop waited for
finishedto be true, and an execution that ends in an error hasfinished: false. It now goes by the executionstatus.
Issue 6: Green, but Merging Is Blocked
GitHub showed “All checks have passed” and, under it:
Merging is blocked
Commits must have verified signatures.
The ruleset on main has required_signatures, and the agent’s commits were
verified: false, reason unsigned. There is no signing key to give a
GitHub App. So the dev runner pushes through the API, and GitHub signs the
commits (Signed commits through the API).
I tested the wrapper on throwaway branches: a new branch, two stacked
branches pushed together, and an amend plus a force push. Every commit came
back verified: true, reason valid. The two #309 commits were replayed the
same way, and CI ran again on the signed copies.
Issue 7: Comments That Said Two Things
I read #309 before the merge, and two of the bot’s comments did not hold up:
- The dry run posted “Review feedback addressed and CI is green” with the note “no unresolved feedback”. It was a test, but on the PR it read like an answer to my review.
- The real run’s comment said “CI is green” in its first line and “CI has not verified 4739f71” a few lines below.
The first is a test copy problem. A test copy of the workflow talks to the real repository, so its comments land on real PRs, with nothing to mark them as a test.
The second is ordering. The runner writes its notes when it finishes. The prompt tells it not to wait for CI, so “CI has not verified” was true when it wrote it. The workflow then polls the checks and wraps those notes in a “CI is green” comment. Two true statements from two different moments, in one comment.
The fix: a test copy starts its comment with “Test run of the implement workflow, not a review reply.”. The runner’s notes leave CI out, and the CI result gets its own line (see Workflow 2). I did not add a second model call to rewrite the summary after CI. Labelling the text with its time was enough.
Results
The reviews
| Run | Focus | Turns | Reported cost | Created |
|---|---|---|---|---|
| 1 | security | not recorded | not recorded | 2 draft advisories, then Issue 2 |
| 2 | security | 74 | 3.33 USD | nothing, Issue 3 |
| 3 | security | 40 | 1.92 USD | 3 draft advisories |
| 4 | performance and scalability | 55 | 2.19 USD | 3 issues, then Issue 4 |
The cost is the API-equivalent price Claude Code reports. On the subscription it is a share of the plan’s usage limit, not a bill.
The security findings are private draft advisories, and they stay out of this post until I fix and publish them. I will fix several of them in one temporary private fork and ship them in one release.
The performance review found three hot paths that hold a lock longer than they need to:
- #306: the ML tree model scores every request inside the rate limiter’s global lock.
- #307: the store
sync drains and sorts up to
max_clientsdeltas while it holds the lock that every request takes. - #308: the cluster heavy hitter check scans the whole shared list on every request under the cluster lock.
The first implementation
I labelled #308 agent:implement at 20:33 and ran the implement workflow:
- The dev runner took 8 turns and reported 0.26 USD. It opened
#309,
fix(engine): index cluster heavy hitters by prefix length, with two commits: the regression test, then the fix (+109, -10). - n8n polled CI three times (about 30 minutes). All 15 check runs passed or
were skipped. #308 went from
agent:in-progresstoagent:greenwith a comment and a Telegram message. - The runner’s notes were worth reading. Nothing was built locally, so CI was
the first compile. The regression test is a timing check (20 000 misses
over 10 000 networks must finish in under 1 s). The verdicts did not
change, so only the speed can fail on the old code. It suggested moving
that gate to the criterion bench if it turns out flaky. It also added a
one-pass constructor, because building the index through the existing
addis quadratic for 10 000 entries.
I reviewed it and added agent:revise, and the next run worked through my
comments.
The first review round
I left four inline comments on #309:
- A test entry (
ttl / 2) that the test set up but never asserted. - The wall-clock limit in the regression test. It can flake (fail at random) on a slow or instrumented runner (coverage), but #308 asked for a test that passes before and after the change.
- A comment in the cluster bench that no longer held after the fix (a miss is now one lookup per prefix length in use, not a scan).
- A doc comment on
Blocksthat still said “blocked by hand (RFC 0014)”, thoughBlocksnow also holds the cluster heavy hitters (RFC 0010).
The second one is the gate that the runner itself suggested moving to the criterion bench “if it turns out flaky”. I did not wait for it to flake.
The run:
| Step | Result |
|---|---|
| Dev runner | 5 turns, 0.16 USD reported |
| Commit | test(engine): assert the half ttl entry, drop the wall-clock gate, signed, verified |
| Threads | a reply on each of the four, all left unresolved for me |
| PR body | the “under 1 s” claim removed, with a PR comment saying so |
| CI | green after three polls, label agent:green, Telegram message |
Its note: the renamed test now checks verdicts only, and the bench
cluster_check_10000 is the speed gate. Nothing was compiled locally, so CI
was again the first build of that commit.
A dry run before my review, with no feedback on the PR, took 2 turns and 0.14 USD and changed nothing. That is the answer I wanted from it.
The merge is mine, from my own agent in the dvarapala checkout.
Lessons
- Give the agents the project’s own rules. dvarapala already had
CLAUDE.md, an RFC template and a stacking skill. When I pointed the agents at them, I got PRs that look like mine, with no new conventions to review. - Keep a human gate on what gets built. A review that files work is cheap. An implementation is a PR that I must read. One label per item keeps that decision with me.
- Route vulnerabilities away from public tracks by construction. The review cannot open a public issue for a security finding. The security branch only knows the advisory endpoint.
- A cached token keeps old permissions. After you change a GitHub App, mint a new token. Anything that renews only on 401 will serve a 403 until the token expires.
- Read the error the tool already printed. The webhook log said “not registered for POST” before I wrote a retry loop.
- Do not cut structured answers with a lazy regex. A model answer that embeds markdown will contain fences. Slice to the last one, or ask for raw JSON.
- Notifications must not fail the work. The message is the least important node in the workflow. It must never be able to stop the rest.
- A bot cannot sign, but GitHub can sign for it. Commits that an App creates through the API are verified. If a ruleset requires signatures, have the agent push through the API instead of looking for a key to give it.
- Let review comments go back to the agent through the same gate. A label I set, feedback only from me, new commits and a reply per thread keep the review readable and the decisions mine.
- Say when a status was true. A summary written before CI and a CI result written after it need their own lines, or the comment contradicts itself. Mark test output as test output wherever it lands.
References
- GitHub: franzramadhan/dvarapala
- GitHub Docs: Generating an installation access token for a GitHub App
- GitHub Docs: Repository security advisories
- GitHub Docs: Collaborating in a temporary private fork
- GitHub Docs: Signature verification for bots
- GitHub Docs: Git database API (blobs, trees, commits, refs)
- GitHub Docs: Rulesets, require signed commits
- GitHub: github/gh-stack
- n8n: Public REST API
- n8n: Webhook node
- Telegram: Bot API formatting options
- Claude Code: Headless mode
- GitHub: wheresryan22/anatomy
