Homelab Rebuild Part 4: Living on the Tailnet

| October 07, 2026 | 6 min to read

Intro

After Part 2 I could reach the cluster only through an SSH bastion, and my Mac couldn’t talk to the LAN from tools like kubectl at all. I wanted the opposite: every admin path over Tailscale, nothing exposed on the internet, and no long-lived cluster credentials on my laptop.

TL;DR: Tailscale SSH for nodes, the Tailscale Kubernetes operator’s API server proxy in auth mode for kubectl (the kubeconfig has token: unused), and Tailscale Ingresses for admin UIs. One gotcha: the Tailscale proxy only speaks TLS to backends on port 443.

The Situation

8 Debian 13 VMs with Tailscale installed but not logged in (Part 2)
Tailscale operator chart 1.102.4
A tailnet with MagicDNS and HTTPS certificates enabled
The old cluster’s devices still registered in the tailnet

Setup

Tailscale SSH on every node

I joined each VM with SSH enabled and without accepting subnet routes, so the VMs don’t reroute LAN traffic that k0s and Cilium depend on:

tailscale up --ssh --hostname=$(hostname)

The tailnet policy’s default ssh rule (autogroup:member to autogroup:self, check mode) already covers devices I own, so this works with no key on the client:

ssh root@k0s-controller-01
# # Authentication checked with Tailscale SSH.

The kubeconfig with no secret

The Tailscale operator can run an API server proxy. In auth mode (apiServerProxyConfig.mode: "true"), the operator impersonates the caller’s tailnet identity, so the kubeconfig carries no token or certificate. Permissions come from a grant in the tailnet policy:

{
  "src": ["group:admin"],
  "dst": ["tag:k8s-operator"],
  "app": {
    "tailscale.com/cap/kubernetes": [
      {"impersonate": {"groups": ["system:masters"]}}
    ]
  }
}

The operator’s OAuth client needs the devices:core, auth_keys and services scopes and the tag:k8s-operator tag. Its credentials go into a Kubernetes Secret created by hand, never into git. Then:

tailscale configure kubeconfig tailscale-operator
kubectl auth whoami
# # Username    me@example.com
# # Groups      [system:masters system:authenticated]

This also solved the Mac problem from Part 2: kubectl talks to a 100.x address, which works for every binary.

Admin UIs as Tailscale Ingresses

OpenBao (Part 5) and Headlamp get ingressClassName: tailscale, which gives each its own https://<name>.<tailnet>.ts.net with a certificate from Tailscale. Headlamp logins use short-lived tokens:

kubectl -n headlamp create token headlamp-admin --duration=12h | pbcopy

Issue 1: Every Device Name Ended in `-1`

Symptom

The new VMs showed up as k0s-controller-01-1, k0s-pve-01-worker-1-1, and so on.

Cause

The old cluster’s devices were still registered and offline, holding the names.

Fix

Delete the stale devices (I used a one-day API access token, then revoked it) and rename the new ones. Same thing happened later with the operator itself (tailscale-operator-1).

Issue 2: The Ingress Returned 400

Symptom

After OpenBao switched to TLS-only (Part 6), its Tailscale Ingress returned 400 on every request.

Cause

The Tailscale proxy sent plain HTTP to OpenBao’s HTTPS listener on port 8200. It only uses TLS toward a backend when the Service port is 443; naming the port https is not enough.

Fix

A small Service that maps 443 to 8200 and selects the active pod, used only by the Ingress:

apiVersion: v1
kind: Service
metadata:
  name: openbao-tailscale
spec:
  selector:
    app.kubernetes.io/name: openbao
    component: server
    openbao-active: "true"
  ports:
    - name: https
      port: 443
      targetPort: 8200

Issue 3: The Chart Gave Headlamp Cluster-Admin

Symptom

Applying my own ClusterRoleBinding named headlamp-admin printed a warning that the object already existed.

Cause

The Headlamp chart creates a ClusterRoleBinding called headlamp-admin by default, binding Headlamp’s own pod to cluster-admin. Headlamp doesn’t need that when users bring their own token.

Fix

clusterRoleBinding.create: false in the chart values, and a separately named binding (headlamp-user-admin) for the login identity.

Update 2026-10-19: The allow-all grant

Next to the Kubernetes grant above, the policy kept Tailscale’s default allow-all grant, {"src": ["*"], "dst": ["*"], "ip": ["*"]}. That was fine while every device on the tailnet was mine. It stopped being fine for one VM I host for a partner: over the tailnet it could reach the cluster API, the nodes and my laptop, even though the Proxmox firewall kept it off the LAN. The src now lists autogroup:member and the two Kubernetes tags, the VM is tag:sandbox, and policy tests deny the sandbox any path in. The full story is in Part 33.

Lessons

Prefer identity over secrets. The tailnet already knows who I am; the kubeconfig doesn’t need to.
Clean up old devices before reusing names. Tailscale won’t reuse a name that an offline device still holds.
Read what a chart creates by default. “Admin UI” charts tend to grant themselves admin.

Series

This post is part of Homelab Rebuild:

Homelab Rebuild Part 4: Living on the Tailnet

Intro

The Situation

Setup

Tailscale SSH on every node

The kubeconfig with no secret

Admin UIs as Tailscale Ingresses

Issue 1: Every Device Name Ended in -1

Symptom

Cause

Fix

Issue 2: The Ingress Returned 400

Symptom

Cause

Fix

Issue 3: The Chart Gave Headlamp Cluster-Admin

Symptom

Cause

Fix

Update 2026-10-19: The allow-all grant

Lessons

Series

References

Issue 1: Every Device Name Ended in `-1`