socialbtw: Hiding Spam in My Threads Replies with a Decision Model

Intro

My Threads posts get the usual replies: friends joking, people asking things, and now and then a hosting reseller or a slot ad. I wanted something that watches my posts, blocks the bad replies and maybe likes the good ones, so I do not have to. It became socialbtw, an open source app that polls my posts, grades every new reply and hides the bad ones.

TL;DR: The Threads API can only hide replies. It has no block, like or delete for a reply from someone else. socialbtw rates each new reply with Cloudflare Clef, a decision model that returns probabilities instead of text. The app hides a reply with P(bad) of 0.85 or more. It sends 0.5 to 0.85 to a chat model for a second opinion, and it keeps anything lower. It is a Rust binary that uses about 1 MiB of memory. It runs on my k0s cluster with its state in CloudNativePG. To get there, I needed a better definition of “spam” and a fix for replies the app could not see. I also hit a Kyverno policy that kept a volume from binding, and a reasoning model that was too slow to be a classifier.

The Figure

Each egg is a reply. It rolls under the candling lamp (the classifier), and the pointer on the panel shows its P(bad) against two fixed stops at 0.5 and 0.85. A clearly bad egg goes into the bin. A clean one rolls on into the carton. An egg between the stops waits under the inspector’s loupe (the reviewer). Click it to grade the next one. The five replies and their scores are real. They come from the first run of the app on my posts. I left out who wrote them. I built the figure with the anatomy skill.

The Situation

• Threads API v1.0 (graph.threads.net), through a Meta app in development mode with my account as the only Threads tester. No app review is needed for your own account.
• Classifier: Cloudflare Clef (@cf/cloudflare/clef, 27B, $0.24 per million input tokens, output free), called through LiteLLM v1.105.1 on its /v1/systemone route (Part 29).
• Reviewer: glm-4.7-flash on Workers AI, through the same LiteLLM virtual key, which is limited to these two models.
• Runtime: the k0s cluster, deployed by Flux (Part 16), state in the CloudNativePG cluster pg (Part 8), secrets from OpenBao through External Secrets (Part 5).
• Alerts: decisions are JSON log lines in VictoriaLogs and metrics in VictoriaMetrics. Grafana sends one Telegram message for each hidden reply and for each reply that needs me (Part 32).
• socialbtw v0.5.0, MIT, on GitHub.

How It Decides

flowchart TD
    R[New reply on my post] --> C["Classifier: Clef<br/>P(bad) = hateful + harassment + spam + sexual"]
    C -- "≥ 0.85" --> H[Hide]
    C -- "0.5 to 0.85" --> V[Reviewer: chat model]
    C -- "< 0.5" --> K[Keep]
    V -- hide --> H
    V -- keep --> K
    V -- unsure --> Q[Telegram: review it yourself]

Clef is not a chat model. It takes a state and a set of typed questions and returns a probability for every allowed answer in one forward pass. socialbtw asks one choice question over six categories (hateful, harassment, spam, sexual, neutral, positive) with the post and the reply as state. Then it adds up the four bad ones. The sum matters. A reply split between hateful and harassment is still clearly bad, even if neither wins alone.

Each reply is decided once and remembered. A reply I unhide in the Threads app comes back with hide_status: UNHUSHED, and the app never hides it again.

The classifier is a Rust trait with two backends. systemone is for decision models in this format (Clef, or TypeSafe Jev). chat is for any OpenAI-compatible chat model, so Claude through LiteLLM works too. The reviewer is always a chat model and only sees the unclear replies.

Issue 1: The API Can Only Hide

Symptom

My plan was “auto block the bad ones, auto like the good ones”. The Threads reply management API has neither.

Cause

For replies to your own posts, the API has POST /{reply-id}/manage_reply?hide=true. It hides a top-level reply and everything nested under it. The API also has reply approvals for posts created through the API. It cannot block a user, like a reply or delete a reply from someone else.

Fix

Hide only, and tell me instead of liking. Grafana sends a Telegram message for every hidden reply and for every reply the reviewer does not decide. Browser automation can block and like, but it breaks the Meta terms and puts the account at risk. I left it out.

Issue 2: Friendly Ads Scored as Positive

Symptom

The first run kept all eight older replies on my posts, including two hosting ads: “pakai .ai.id di anekahost.com cuma 80.000 pertahun kak” scored 0.24 and “pake (dot)ai(dot)id aja bang, di idwebhost cuma Rp 99.000” scored 0.04. Clef put both in positive.

Cause

My spam definition said “promotion, scams, crypto, links to unrelated sites”. A friendly, on-topic recommendation with a price, under a post about domain prices, read as helpful. And positive said nothing against promotion.

Fix

I rewrote the definitions. spam now names product and service promotion with a price or offer, domains and links (also obfuscated like (dot)), and referral codes. It also adds “a reply can be friendly and on-topic and still be spam”. neutral and positive now exclude promotion. I compared both versions on the same replies before I shipped. I used a stand-in post text, so the “before” column differs from the first run:

In production, with the real post text, the two ads came back at 0.90 (hidden directly) and 0.79. The 0.79 reply went to the reviewer. Its answer was “Promoting a specific product with a price.” Both are hidden now.

Issue 3: The Replies Were There, the App Did Not See Them

Symptom

The first deploy found my 21 posts from the last three days and 0 replies on every one of them. But I could see replies in the app.

Cause

Two things. 19 of the 21 were reposts (media_type: REPOST_FACADE). They show up in /me/threads, but their replies belong to the thread of the original author. And my own posts from those three days really had no replies yet. The nine replies were on five posts from 24 September to 9 October, outside the default LOOKBACK_DAYS=3.

Fix

Skip reposts, and read 30 days back. A read of every post every five minutes then costs one API call per post per poll. So the app reads quiet posts less often, by the age of their last activity:

A new reply makes a post count as fresh again. Later I also limited it to posts with the topic yapping or the hashtag #yapping. That filter is plain string matching before the app reads any reply. Other posts cost no reply calls and no model calls at all.

Issue 4: The Volume Never Bound

Symptom

The first version kept its state in SQLite on a 1 Gi local-path volume. After Flux applied it, the PersistentVolume was Bound, but the claim stayed Pending. The pod stayed Pending with a nominated node and no events.

Cause

The kube-controller-manager journal in VictoriaLogs had the answer, every 15 seconds:

"Could not sync claim" err="admission webhook \"vpol.validate.kyverno.svc-ignore-...\" denied the request:
Policy protect-flux-managed failed: Flux-managed object changed outside git by
system:serviceaccount:kube-system:persistent-volume-binder

protect-flux-managed (Part 11) denies changes to Flux-managed objects by anyone but the Flux controllers. A claim that Flux applies directly carries the Flux labels. To bind it, the volume binder writes spec.volumeName into it. kubectl --dry-run=server did not reproduce it, because dry runs skip that webhook.

Fix

A StatefulSet with a volumeClaimTemplates claim. The StatefulSet controller creates that claim, so it has no Flux labels. A few releases later, I moved the state to a socialbtw database on the CloudNativePG cluster, with TLS checked against the lab CA. The app became a plain Deployment with no volume. It now runs on whichever worker has room.

Issue 5: A Reasoning Model Is a Slow Classifier

Symptom

To keep the classifier swappable, I added a chat backend and tested it with glm-4.7-flash. The test failed with Network Error after exactly 60 seconds, which is the HTTP timeout of the app. It failed again after 120.

Cause

glm-4.7-flash reasons before it answers. One classification took 32 seconds and 880 completion tokens. Another took 43 seconds and 2,382 tokens. Some took longer than two minutes. The answers were right (the hosting ad came back as spam at 0.9), but slow. Clef returns its probabilities in one forward pass, without generating any text.

Fix

I added HTTP_TIMEOUT_SECONDS (default 120) and an error message that says timed out instead of nothing. Clef stays the classifier. A fast non-reasoning model is the better choice for the chat backend. The reviewer is still glm-4.7-flash, because it only sees the few unclear replies.

Keeping It Running

flowchart LR
    T[Threads API] <--> S[socialbtw<br/>k0s Deployment]
    S --> L[LiteLLM] --> W[Workers AI:<br/>Clef, glm-4.7-flash]
    S <--> DB[(CloudNativePG<br/>socialbtw)]
    S --> O[VictoriaLogs and<br/>VictoriaMetrics] --> GR[Grafana] --> TG[Telegram]
    B[My browser on the LAN] --> GW[lab-gateway<br/>/oauth/ only] --> S

• The token never expires while the app runs. The Threads long-lived token lasts 60 days. The app swaps it for a new one every day with refresh_access_token and keeps it in the database.
• A new token takes one link. /oauth/start sends me to Threads to approve. /oauth/callback exchanges the code, checks that the token belongs to my account and stores it. The route is a Gateway API HTTPRoute on lab-gateway for socialbtw.franzramadhan.lab, with only /oauth/ routed. It needs no public exposure, because the redirect happens in my browser.
• Rate limits back off. HTTP 429 or Meta error codes 4, 17, 32 and 613 pause polling for 15 minutes. The pause doubles each time, up to 6 hours. The app keeps a failed hide as hide_pending and retries it without a new classification, so a retry costs no model calls.
• It stays small. I rewrote the first Python version in Rust. It is a static musl binary in a scratch image of 5.2 MB, and it uses 1.2 MiB of memory in a container. The arm64 image build under QEMU took 12.5 minutes. Native arm64 runners brought a release down to about 4 minutes.
• It started in dry run. At first the app only logged every decision as would_hide. I checked those alerts in Telegram before I switched DRY_RUN off.

Lessons

1. Read the API before designing the feature. “Block and like” became “hide and notify” in the first hour, and that shaped everything after it.
2. A classifier is only as good as its definitions. The model was right about what I described. I described spam too narrowly.
3. Zero results needs a second look. No replies on 21 posts was really 19 reposts and a three-day window.
4. A guardrail can block the system, not only people. The Flux protection policy stopped the volume binder of Kubernetes itself. The controller-manager log was the only place that said so.
5. Pick the model for the job. A decision model answers in one pass. A reasoning model is good for the few unclear cases and too slow for all of them.

References

• GitHub: franzramadhan/socialbtw
• GitHub: wheresryan22/anatomy
• Meta for Developers: Threads reply management
• Meta for Developers: Threads get access tokens and permissions
• Cloudflare: Clef on Workers AI
• LiteLLM: Decisions API
• Kubernetes: StatefulSets
• CloudNativePG: Declarative database management