socialbtw: Hiding Spam in My Threads Replies with a Decision Model

Intro

My Threads posts get the usual replies: friends joking, people asking things, and now and then a hosting reseller or a slot ad. I wanted something that watches my posts, blocks the bad replies and maybe likes the good ones, so I don’t have to. It became socialbtw, an open source app that polls my posts, grades every new reply and hides the bad ones.

TL;DR: The Threads API can only hide replies: there is no block, like or delete for someone else’s reply. socialbtw rates each new reply with Cloudflare Clef, a decision model that returns probabilities instead of text. P(bad) of 0.85 or more is hidden, 0.5 to 0.85 goes to a chat model for a second opinion, and anything lower is kept. It is a Rust binary using about 1 MiB of memory, deployed on my k0s cluster with its state in CloudNativePG. Getting there took a better definition of “spam”, a fix for replies the app could not see, a Kyverno policy that kept a volume from binding, and a reasoning model that was too slow to be a classifier.

The Figure

Each egg is a reply. It rolls under the candling lamp (the classifier), and the pointer on the panel shows its P(bad) against two fixed stops at 0.5 and 0.85. A clearly bad egg is pushed into the bin, a clean one rolls on into the carton, and one between the stops waits under the inspector’s loupe (the reviewer). Click it to grade the next one. The five replies and their scores are real, from the app’s first run on my posts; I left out who wrote them. The figure was built with the anatomy skill.

The Situation

• Threads API v1.0 (graph.threads.net), through a Meta app in development mode with my account as the only Threads tester. No app review is needed for your own account.
• Classifier: Cloudflare Clef (@cf/cloudflare/clef, 27B, $0.24 per million input tokens, output free), called through LiteLLM v1.105.1 on its /v1/systemone route (Part 29 ).
• Reviewer: glm-4.7-flash on Workers AI, through the same LiteLLM virtual key, which is limited to these two models.
• Runtime: the k0s cluster, deployed by Flux (Part 16 ), state in the CloudNativePG cluster pg (Part 8 ), secrets from OpenBao through External Secrets (Part 5 ).
• Alerts: decisions are JSON log lines in VictoriaLogs and metrics in VictoriaMetrics; Grafana sends one Telegram message per hidden reply or reply that needs me (Part 32 ).
• socialbtw v0.5.0, MIT, on GitHub .

How It Decides

flowchart TD
    R[New reply on my post] --> C["Classifier: Clef<br/>P(bad) = hateful + harassment + spam + sexual"]
    C -- "≥ 0.85" --> H[Hide]
    C -- "0.5 to 0.85" --> V[Reviewer: chat model]
    C -- "< 0.5" --> K[Keep]
    V -- hide --> H
    V -- keep --> K
    V -- unsure --> Q[Telegram: review it yourself]

Clef is not a chat model. It takes a state and a set of typed questions and returns a probability for every allowed answer in one forward pass. socialbtw asks one choice question over six categories (hateful, harassment, spam, sexual, neutral, positive) with the post and the reply as state, and adds up the four bad ones. Summing matters: a reply split between hateful and harassment is still clearly bad even if neither wins alone.

Each reply is decided once and remembered. A reply I unhide in the Threads app comes back with hide_status: UNHUSHED, and the app never hides it again.

The classifier is a trait with two backends: systemone for decision models in this format (Clef, or TypeSafe Jev) and chat for any OpenAI-compatible chat model, so Claude through LiteLLM works too. The reviewer is always a chat model and only sees the unclear replies.

Issue 1: The API Can Only Hide

Symptom

My plan was “auto block the bad ones, auto like the good ones”. The Threads reply management API has neither.

Cause

For replies to your own posts there is POST /{reply-id}/manage_reply?hide=true, which hides a top-level reply and everything nested under it, and there are reply approvals for posts created through the API. Blocking a user, liking a reply or deleting someone else’s reply is not in the API.

Fix

Hide only, and tell me instead of liking: Grafana sends a Telegram message for every hidden reply and for every reply the reviewer would not decide. Browser automation could block and like, but it breaks Meta’s terms and risks the account, so I left it out.

Issue 2: Friendly Ads Scored as Positive

Symptom

The first run kept all eight older replies on my posts, including two hosting ads: “pakai .ai.id di anekahost.com cuma 80.000 pertahun kak” scored 0.24 and “pake (dot)ai(dot)id aja bang, di idwebhost cuma Rp 99.000” scored 0.04. Clef put both in positive.

Cause

My spam definition said “promotion, scams, crypto, links to unrelated sites”. A friendly, on-topic recommendation with a price, under a post about domain prices, read as helpful, and positive said nothing against promotion.

Fix

I rewrote the definitions: spam now names product and service promotion with a price or offer, domains and links (also obfuscated like (dot)), referral codes, and adds “a reply can be friendly and on-topic and still be spam”; neutral and positive now exclude promotion. I compared both versions on the same replies before shipping (with a stand-in post text, so the “before” column differs from the first run):

In production, with the real post text, the two ads came back at 0.90 (hidden directly) and 0.79, which went to the reviewer, whose answer was “Promoting a specific product with a price.” Both are hidden now.

Issue 3: The Replies Were There, the App Did Not See Them

Symptom

The first deploy found my 21 posts from the last three days and 0 replies on every one of them, though I could see replies in the app.

Cause

Two things. 19 of the 21 were reposts (media_type: REPOST_FACADE): they show up in /me/threads, but their replies belong to the original author’s thread. And my own posts from those three days really had no replies yet; the nine replies were on five posts from 24 September to 9 October, outside the default LOOKBACK_DAYS=3.

Fix

Skip reposts, and read 30 days back. Reading every post every five minutes then costs one API call per post per poll, so quiet posts are read less often, by the age of their last activity:

A new reply makes a post count as fresh again. Later I also limited it to posts with the topic yapping or the hashtag #yapping; that filter is plain string matching before any reply is read, so other posts cost no reply calls and no model calls at all.

Issue 4: The Volume Never Bound

Symptom

The first version kept its state in SQLite on a 1 Gi local-path volume. After Flux applied it, the PersistentVolume was Bound but the claim stayed Pending, and the pod stayed Pending with a nominated node and no events.

Cause

The kube-controller-manager journal in VictoriaLogs had the answer, every 15 seconds:

"Could not sync claim" err="admission webhook \"vpol.validate.kyverno.svc-ignore-...\" denied the request:
Policy protect-flux-managed failed: Flux-managed object changed outside git by
system:serviceaccount:kube-system:persistent-volume-binder

protect-flux-managed (Part 11 ) denies changes to Flux-managed objects by anyone but the Flux controllers. A claim applied directly by Flux carries Flux’s labels, and binding it means the volume binder writes spec.volumeName into it. kubectl --dry-run=server did not reproduce it, because dry runs skip that webhook.

Fix

A StatefulSet with a volumeClaimTemplates claim: the StatefulSet controller creates that claim, so it has no Flux labels. A few releases later I moved the state to a socialbtw database on the CloudNativePG cluster, with TLS checked against the lab CA, and the app became a plain Deployment with no volume. It now runs on whichever worker has room.

Issue 5: A Reasoning Model Is a Slow Classifier

Symptom

To keep the classifier swappable I added a chat backend and tested it with glm-4.7-flash. The test failed with Network Error after exactly 60 seconds, the app’s HTTP timeout, and again after 120.

Cause

glm-4.7-flash reasons before it answers. One classification took 32 seconds and 880 completion tokens, another 43 seconds and 2,382 tokens, and some took longer than two minutes. The answers were right (the hosting ad came back as spam at 0.9), just slow. Clef returns its probabilities in one forward pass, without generating any text.

Fix

HTTP_TIMEOUT_SECONDS (default 120), an error message that says timed out instead of nothing, and Clef stays the classifier. A fast non-reasoning model is the better choice for the chat backend. The reviewer is still glm-4.7-flash, because it only sees the few unclear replies.

Keeping It Running

flowchart LR
    T[Threads API] <--> S[socialbtw<br/>k0s Deployment]
    S --> L[LiteLLM] --> W[Workers AI:<br/>Clef, glm-4.7-flash]
    S <--> DB[(CloudNativePG<br/>socialbtw)]
    S --> O[VictoriaLogs and<br/>VictoriaMetrics] --> GR[Grafana] --> TG[Telegram]
    B[My browser on the LAN] --> GW[lab-gateway<br/>/oauth/ only] --> S

• The token never expires while the app runs. Threads’ long-lived token lasts 60 days; the app swaps it for a new one every day with refresh_access_token and keeps it in the database.
• Getting a new token is one link. /oauth/start sends me to Threads to approve, /oauth/callback exchanges the code, checks the token belongs to my account and stores it. The route is a Gateway API HTTPRoute on lab-gateway for socialbtw.franzramadhan.lab, with only /oauth/ routed. It needs no public exposure, because the redirect happens in my browser.
• Rate limits back off. HTTP 429 or Meta error codes 4, 17, 32 and 613 pause polling for 15 minutes, doubling up to 6 hours. A hide that fails is kept as hide_pending and retried without classifying the reply again, so a retry costs no model calls.
• It stays small. I rewrote the first Python version in Rust: a static musl binary in a scratch image of 5.2 MB, using 1.2 MiB of memory in a container. Building the arm64 image under QEMU took 12.5 minutes; native arm64 runners brought a release down to about 4 minutes.
• It started in dry run. At first every decision was only logged as would_hide; I checked those alerts in Telegram before switching DRY_RUN off.

Lessons

1. Read the API before designing the feature. “Block and like” became “hide and notify” in the first hour, and that shaped everything after it.
2. A classifier is only as good as its definitions. The model was right about what I described; I had described spam too narrowly.
3. Zero results needs a second look. No replies on 21 posts was really 19 reposts and a three-day window.
4. A guardrail can block the system, not only people. The Flux protection policy stopped Kubernetes’ own volume binder; the controller-manager log was the only place that said so.
5. Pick the model for the job. A decision model answers in one pass; a reasoning model is great for the few unclear cases and too slow for all of them.

References

• GitHub: franzramadhan/socialbtw
• GitHub: wheresryan22/anatomy
• Meta for Developers: Threads reply management
• Meta for Developers: Threads get access tokens and permissions
• Cloudflare: Clef on Workers AI
• LiteLLM: Decisions API
• Kubernetes: StatefulSets
• CloudNativePG: Declarative database management