Homelab Rebuild Part 5: Secrets Without Secrets in Git

| October 08, 2026 | 6 min to read

Intro

The old repo had a Pi-hole admin password and a Tailscale OAuth client secret committed in plain text. For the rebuild, the rule was simple: no credential in git, ever. That needs somewhere else to keep them.

TL;DR: OpenBao 2.7.1 as a 3-node Raft cluster with static auto-unseal, External Secrets Operator to turn OpenBao entries into Kubernetes Secrets, and the agent injector for workloads that should never have a Secret object. Every secret enters through stdin, never through arguments or the terminal.

The Figure

What moves: a secret on its way outRoof: openbao-0, 1, 2 (left to right), time lock, seal keyOpenBao 2.7.1 runs as three Raft members, one per Proxmox host. The static seal reads its key from a mounted Kubernetes Secret, so a restarted member unseals itself without anyone typing a key share. External Secrets Operator logs in with Kubernetes auth and a read-only policy and files each entry as a Kubernetes Secret; the agent injector writes straight into the pod at /vault/secrets with no Secret object. No credential is in Git.

The vault is OpenBao. The three lock stations on its roof are the Raft members, one per Proxmox host, and the time lock beside them is the static seal: press 1, 2 or 3 to restart a member and watch it unseal itself from the key box, with nobody typing a key share. E sends the courier lift (External Secrets Operator) to fetch a sealed envelope and file it in the rack of Kubernetes Secrets. I fires the pneumatic tube (the agent injector), which writes into the pod and never touches the rack. I generated the figure with the anatomy agent skill.

The Situation

OpenBao chart 0.30.3 (OpenBao 2.7.1)
External Secrets Operator 2.12.0
local-path-provisioner v0.0.37 as the StorageClass
No Longhorn (I had removed it). Raft replicates OpenBao’s data itself, so node-local disks are enough.

Setup

OpenBao on Raft

Three server pods, one per Proxmox host, joined through retry_join:

storage "raft" {
  path = "/openbao/data"
  retry_join { leader_api_addr = "https://openbao-0.openbao-internal:8200" }
  retry_join { leader_api_addr = "https://openbao-1.openbao-internal:8200" }
  retry_join { leader_api_addr = "https://openbao-2.openbao-internal:8200" }
}

The StatefulSet is OrderedReady, so openbao-1 and openbao-2 start only after openbao-0 is initialized and unsealed.

Static auto-unseal, and its trade-off

I didn’t want to type three key shares after every pod restart, and I had no KMS handy. OpenBao’s static seal reads a 32-byte key from a file:

seal "static" {
  current_key_id = "lab-2026-10-12"
  current_key    = "file:///openbao/userconfig/openbao-seal-key/key"
}

The key lives in a Kubernetes Secret mounted into the pods, with a backup copy outside the cluster. The trade-off is explicit: anyone with cluster-admin can read the unseal key. For a single-person homelab that’s acceptable; for anything shared it isn’t.

With auto-unseal, bao operator init returns recovery keys instead of unseal keys. I wrote the output straight to a mode-600 file instead of the terminal:

(umask 077; kubectl -n openbao exec openbao-0 -- \
  bao operator init -recovery-shares=5 -recovery-threshold=3 -format=json > init.json)

Secrets go in through stdin

Every write to OpenBao follows the same pattern: the token and the value travel through stdin, so they never appear in process arguments, shell history or the terminal:

{ jq -r .root_token init.json; printf %s "$VALUE"; } \
  | kubectl -n openbao exec -i openbao-0 -- sh -c \
    'read -r BAO_TOKEN; export BAO_TOKEN; bao kv put -mount=secret my-app/db password=-'

External Secrets Operator

ESO authenticates to OpenBao with Kubernetes auth and a read-only policy, and apps declare what they need:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: pihole-admin
spec:
  secretStoreRef:
    kind: ClusterSecretStore
    name: openbao
  target:
    name: pihole-admin
  data:
    - secretKey: password
      remoteRef:
        key: pihole/admin
        property: password

Today this feeds the Pi-hole password, the Cloudflare token for cert-manager and external-dns, and the Cloudflare Tunnel token.

Issue 1: The Injector Ignored My Pod

Symptom

A test pod with openbao.hashicorp.com/agent-inject: "true" started without any sidecar.

Cause

The chart’s injector image is still hashicorp/vault-k8s, which watches the vault.hashicorp.com/ annotation prefix. The injected agent itself is OpenBao (quay.io/openbao/openbao).

Fix

Use the vault.hashicorp.com/ annotations. The secret landed in /vault/secrets/greeting as expected.

Issue 2: Helm Wouldn’t Upgrade the Chart

Symptom

UPGRADE FAILED: conflict occurred while applying object /openbao-agent-injector-cfg ...
conflict with "vault-k8s": .webhooks[name="vault.hashicorp.com"].clientConfig.caBundle

Cause

Helm 4 applies server-side. The injector writes its own webhook caBundle at runtime, so it owns that field.

Fix

helm upgrade --force-conflicts. The injector rewrites the caBundle right away.

Lessons

Decide the unseal trade-off on purpose. Static seal buys unattended restarts by putting the key next to the data.
stdin is the secret channel. Arguments end up in ps, history and logs.
Forks keep their parents’ names in odd places. OpenBao’s injector still answers to vault.hashicorp.com/.

Series

This post is part of Homelab Rebuild:

Homelab Rebuild Part 5: Secrets Without Secrets in Git

Intro

The Figure

The Situation

Setup

OpenBao on Raft

Static auto-unseal, and its trade-off

Secrets go in through stdin

External Secrets Operator

Issue 1: The Injector Ignored My Pod

Symptom

Cause

Fix

Issue 2: Helm Wouldn’t Upgrade the Chart

Symptom

Cause

Fix

Lessons

Series

References