Back to Home
post-thumb

Homelab Rebuild Part 30: Proxmox Joins the SSO, and a Plan Token Loses Admin

| October 18, 2026 | 9 min to read

Intro

OpenBao (Part 19) and Headlamp (Part 20) already log me in through Cloudflare Access and Google. The Proxmox UI was the odd one out: root@pam and a password, on a page my browser flagged as “Not Secure” because port 8006 served a certificate from Proxmox’s own CA.

While checking the access list for the new login, I found a second problem: the token Terraform uses for plans had full Administrator rights.

TL;DR: Proxmox VE 9 has an OpenID realm type. One realm cf pointing at a Cloudflare Access SaaS app covers all three nodes, with one pre-created user and no auto-creation. tailscale cert plus pvenode cert set gives each node a Let’s Encrypt certificate for its tailnet name, renewed by a weekly timer. The plan token lost Administrator, but read-only was not enough: it needs VM.Config.Disk too.

The Figure

ACCESSUSERS01root@pam02Grealm cf03AUDIT x7DISK
Key card or master key in useDoor lock: one Proxmox nodeThe ID reader is Cloudflare Access: Google login, and a policy that allows one email. The card file is the list of pre-created Proxmox users, with one card in it, because the realm has autocreate off. The encoder is the OpenID realm cf. It cuts a key card only for a guest who passed the reader and has a card in the file. The realm lives in the cluster file system, so one card opens 01, 02 and 03. root@pam is the master key behind glass, for when Cloudflare is down. The rail in front is the plan token after the cleanup: seven audit keys, one disk key (VM.Config.Disk, which the provider needs to read disk volumes) and an empty hook where Administrator hung. 1 is the owner, 2 another Google account, 3 breaks the glass, 4 shows the rail; a click steps through them.

The figure is a hotel key desk, because that is what the new login is. The ID reader at the left is Cloudflare Access: it checks the guest’s Google badge and lights pass or deny. Behind it, the card file holds the pre-created users, and there is one card in it. The encoder in the middle is the realm cf. It cuts a key card only for a guest who passed the reader and has a card in the file, and that one card opens all three doors, pve-01 to pve-03. The master key behind glass at the back is root@pam. The rail in front is the plan token after Issue 1: seven audit keys, one disk key and an empty hook. Press 1 for my login, 2 for another Google account, 3 to break the glass and 4 for the rail, or click to step through them. I built it with the anatomy skill.

The Situation

  • Proxmox VE 9.2.21 on pve-01, pve-02 and pve-03, one cluster
  • Realms before: pam and pve only
  • I open the UI at https://pve-0X.<tailnet>.ts.net:8006, never by LAN IP
  • Cloudflare Zero Trust team <team>.cloudflareaccess.com, Google as the identity provider, same as Parts 19 and 20
  • Terraform (bpg/proxmox) runs in CI with two API tokens: terraform-plan@pve!github and terraform-apply@pve!github

Setup

The login flow

sequenceDiagram
  participant B as Browser
  participant P as pveproxy :8006
  participant C as Cloudflare Access
  participant G as Google
  B->>P: Login, realm cf
  P-->>B: redirect to issuer (redirect_uri = browser origin)
  B->>C: authorize
  C->>G: sign in
  G-->>C: identity
  C-->>B: redirect back with code
  B->>P: code
  P->>C: exchange code for ID token
  P-->>B: session as <email>@cf

Proxmox talks to Cloudflare only for the code exchange. Everything else goes through the browser, which matters for the redirect URLs below.

Step 1: The Cloudflare side

In Zero Trust: Access, Applications, Add an application, SaaS, protocol OIDC. Scopes openid, email, profile. Login method Google, and a policy that allows only my email address.

The redirect URLs need care. Proxmox sends the browser’s origin as the redirect_uri, so every URL I type into the address bar has to be listed, exactly, with the port and without a path:

https://pve-01.<tailnet>.ts.net:8006
https://pve-02.<tailnet>.ts.net:8006
https://pve-03.<tailnet>.ts.net:8006

Before using the issuer URL, I checked its discovery document:

curl -s "$ISSUER/.well-known/openid-configuration" \
  | python3 -c "import json,sys;d=json.load(sys.stdin);print(d['issuer']==\"$ISSUER\", d['scopes_supported'])"
# True ['openid', 'email', 'profile']

The client ID and secret went straight into OpenBao at secret/proxmox/oidc, entered at a silent read -s prompt, so they never showed up in a terminal or a chat.

Step 2: The realm

Realm config lives in /etc/pve, which the cluster file system replicates, so one command on pve-01 covers all three nodes. The secret comes from OpenBao over SSH stdin:

bao kv get -format=json secret/proxmox/oidc \
  | python3 -c "import json,sys;d=json.load(sys.stdin)['data']['data'];print(d['client_id']);print(d['client_secret'])" \
  | ssh root@pve-01 'read -r CID; read -r CSEC
      pveum realm add cf --type openid --issuer-url "$ISSUER" \
        --client-id "$CID" --client-key "$CSEC" \
        --username-claim email --scopes "openid email profile" \
        --autocreate 0 --comment "Cloudflare Access (Google)"'
  • --username-claim email makes the user <email>@cf instead of an opaque subject ID.
  • --autocreate 0: a valid Google login is not enough. The user must exist.

Then the one user and its rights:

pveum user add <email>@cf --comment "SSO via Cloudflare Access"
pveum acl modify / --users <email>@cf --roles Administrator

My Claude Code session prepared that last command, but its permission classifier refused to grant Administrator on its own, so I ran it myself. That’s the right call for a command that creates a new full admin.

root@pam stays as the break-glass login. API tokens and SSH don’t touch the realm, so Terraform, the exporter and qm over SSH work as before.

Step 3: Real certificates on port 8006

Each node already had Tailscale HTTPS certificates enabled for its tailnet name. A small script fetches the certificate and hands it to Proxmox:

tailscale cert --cert-file "$dir/cert.pem" --key-file "$dir/key.pem" "$name"
if ! cmp -s "$dir/cert.pem" /etc/pve/local/pveproxy-ssl.pem; then
  pvenode cert set "$dir/cert.pem" "$dir/key.pem" --force 1 --restart 1
fi

tailscale cert reuses the cached certificate until it gets close to expiry, and pveproxy restarts only when the file changed. A systemd timer runs it every Monday at 10:00 WIB. Let’s Encrypt certificates last 90 days, so a weekly run leaves plenty of retries.

Things I checked before rolling it past pve-03:

  • Right after the restart, openssl s_client still showed the old Proxmox CA certificate. A few seconds later it showed Let’s Encrypt. The restart is not instant.
  • Requests proxied between nodes still work: pvesh get /nodes/pve-03/status from pve-01 answered after the swap.
  • Terraform (insecure = true) and the PVE exporter (verify_ssl: false) connect by LAN IP and skip verification, so the new certificate can’t break them.

Result

NodeRealm cfCert timer:8006 certificate
pve-01presentenabledLet’s Encrypt, verifies
pve-02presentenabledLet’s Encrypt, verifies
pve-03presentenabledLet’s Encrypt, verifies

Login with realm cf works on all three URLs. Chrome kept showing “Not Secure” for a while: it remembered the exception I had clicked through for the old self-signed certificate. A browser restart cleared it, and curl confirmed the chain was fine all along (ssl_verify_result 0).

Issue 1: A plan token with Administrator

Symptom

pveum acl list while adding the new user:

/  Administrator  user  terraform-apply@pve
/  Administrator  user  terraform-plan@pve
/  PVEAuditor     user  terraform-plan@pve

The token that only runs terraform plan on pull requests could create, change or delete anything in the cluster.

Cause

A leftover from Part 7. The old Telmate provider failed with a missing VM.Monitor privilege, and Administrator was added while chasing that error. It didn’t help: Proxmox VE 9 removed VM.Monitor, so not even Administrator has it. A provider bump fixed the error, and the extra role was never taken back. The tokens use privsep=0, so each one has exactly its user’s rights.

First try: read-only

The plan only reads VMs, storage and the guest agent’s network data, so PVEAuditor alone looked like enough:

pveum acl delete / --users terraform-plan@pve --roles Administrator
pveum user permissions terraform-plan@pve --path /vms/101
# Datastore.Audit, Mapping.Audit, Pool.Audit, SDN.Audit,
# Sys.Audit, VM.Audit, VM.GuestAgent.Audit

To prove it, I opened a comment-only pull request so CI would run a real plan. It failed on every VM disk:

Error: error get file local-lvm:vm-401-disk-0 from datastore local-lvm:
received an HTTP 403 response - Reason: Permission check failed
(/vms/401, VM.Config.Disk)

The bpg provider reads each disk volume during refresh. For a volume that belongs to a VM, Proxmox checks VM.Config.Disk on that VM, a privilege meant for changing disks. No read-only role includes it.

Fix

A custom role with the auditor privileges plus that one:

pveum role add TerraformPlan --privs "Datastore.Audit,Mapping.Audit,Pool.Audit,SDN.Audit,Sys.Audit,VM.Audit,VM.GuestAgent.Audit,VM.Config.Disk"
pveum acl modify / --users terraform-plan@pve --roles TerraformPlan
pveum acl delete / --users terraform-plan@pve --roles PVEAuditor

The re-run plan passed with No changes.

It’s not read-only: the plan token can still add, detach or resize VM disks. But it can no longer create or delete VMs, power them on or off, or touch users and tokens. The apply token keeps Administrator for now.

Lessons

  1. OIDC redirect URLs are browser origins. Proxmox builds the redirect_uri from the address bar, so list every name you actually type, port included.
  2. Turn off auto-create. With one pre-created user, a valid login at the identity provider still isn’t enough to get into the cluster.
  3. Tailscale already issues real certificates. If you reach a host by its tailnet name, tailscale cert is the shortest way to drop the self-signed warning.
  4. Prove least privilege with a real run. “A plan only reads” was wrong for this provider; the CI plan found the missing privilege in a minute.
  5. Read the ACL list when you touch auth. The over-privileged plan token had been sitting there for six days, and I only saw it because I was adding a user next to it.

Series

This post is part of Homelab Rebuild:

  1. Part 1: Upgrading Proxmox VE 8 to 9
  2. Part 2: Starting Over When the IaC Is Broken
  3. Part 3: When Tailscale Breaks Your Cilium Gateway
  4. Part 4: Living on the Tailnet
  5. Part 5: Secrets Without Secrets in Git
  6. Part 6: Certificates, DNS and a Private CA
  7. Part 7: Self-Hosted CI and Getting Back to IaC
  8. Part 8: Stateful Workloads on Unequal Hardware
  9. Part 9: Going Public Safely with Cloudflare Tunnel
  10. Part 10: When the Hypervisor Went Quiet
  11. Part 11: Hardening the Cluster in Five Layers
  12. Part 12: Observability with Coroot, and Losing pve-01
  13. Part 13: The Host That Kept Losing Power
  14. Part 14: Observability, Take Two with OpenTelemetry
  15. Part 15: Requests and Limits for Everything
  16. Part 16: Handing the Cluster to FluxCD
  17. Part 17: Making the Guardrails Earn Their Keep
  18. Part 18: When the API Server Outgrew Its VM
  19. Part 19: Logging In to OpenBao with Cloudflare Access and Google
  20. Part 20: Headlamp SSO, or Teaching kube-apiserver to Trust Cloudflare Access
  21. Part 21: pve-02 Loses Power, and Pi-hole Has Nowhere to Go
  22. Part 22: Turning Four Days of Fixes into AI Runbooks
  23. Part 23: microVM Sandboxes on Proxmox
  24. Part 24: Pi-hole Without a Home Host
  25. Part 25: It Was the Router
  26. Part 26: TLS on the Front, Plaintext Behind It
  27. Part 27: One Bad Power Adapter, Two Different Deaths
  28. Part 28: Observability, Take Three: Making It Worth Looking At
  29. Part 29: One Meter for Every Model, LiteLLM on Workers AI
  30. Part 30: Proxmox Joins the SSO, and a Plan Token Loses Admin (this post)
  31. Part 31: First Backups, Postgres, OpenBao and etcd to R2

References