Back to Home
post-thumb

Homelab Rebuild Part 39: Wasm Pods on k0s, and the Label That Kept Them Packed

| October 26, 2026 | 11 min to read

Intro

I want to run some of my own small services as WebAssembly modules on the k0s cluster, next to the normal containers. A Mirantis blog post walks through exactly that on k0s. You build the runwasi shims (small helper processes that let containerd start Wasm modules). Then you drop a containerd config file into /etc/k0s/containerd.d/, add a RuntimeClass, and you are done. So I had two questions: does the guide work on my cluster, and which Wasm runtime do I pick?

The guide’s mechanism is right, but two things in it do not work on a current k0s. The second one cost me most of a morning.

TL;DR: use the prebuilt, signed runwasi Wasmtime shim and a containerd drop-in (an extra config file that containerd imports). On k0s 1.36 (containerd 2.x), the runtime goes under io.containerd.cri.v1.runtime, not the containerd 1.x path that most guides show. Also build your Wasm images as linux/amd64, not wasip1/wasm. Kubelet pulls with the host platform. So it pulls a wasip1/wasm image but never unpacks it, and the pod dies with parent snapshot ... does not exist.

The Figure

containerd 2.3 · config v3
Tonearms: the runtime handler, picked by runtimeClassName (back runc, front wasmtime)The changer: kubelet and CRI pull with the default platform, so only a linux/amd64 record reaches the platterThe card in the slot: the drop-in /etc/k0s/containerd.d/wasmtime.toml; the greyed switch: RuntimeClassInImageCriAPI (alpha, off)Each record is an image and its label is the platform. The changer is kubelet and CRI: they pull with the default platform and unpack only a linux/amd64 image into a snapshot, the platter. The upstream wasi-demo-app is wasip1/wasm, so it pulls but never lands: CreateContainerError, parent snapshot does not exist. Rebuilt as a linux/amd64 scratch image (636 KB) it drops onto the platter. The wasmtime arm (runwasi containerd-shim-wasmtime-v1 v0.6.1, registered by the drop-in card) plays it and the pod prints hello from wasmtime on k0s and reaches Completed. The runc arm tries to run the same .wasm file as a Linux binary: exec format error.

The figure is a jukebox drawn as a record changer. Each record is an image, and its label is the platform. The carriage on the overhead rail is kubelet and CRI. They pull with the default platform, so the carriage only lifts a linux/amd64 record onto the platter (the unpacked snapshot). A wasip1/wasm record stays in the magazine and the lamp beside it lights, because the image pulled but never unpacked. Two tonearms stand at the right of the platter: runc at the back and wasmtime in front. runtimeClassName picks one. The card in the slot is the drop-in wasmtime.toml. The small greyed switch is RuntimeClassInImageCriAPI, still alpha and off. Press P or click the record to flip the platform. Click an arm (or press R/W) to pick the runtime. wasip1/wasm fails with parent snapshot does not exist on either arm. linux/amd64 on wasmtime prints hello from wasmtime on k0s and reaches Completed. linux/amd64 on runc gets exec format error. Built with the anatomy skill.

The Situation

  • k0s v1.36.4+k0s.1 on 5 Debian 13 workers and 3 controllers, data dir /opt/lib/k0s. Workers run the k0sworker service.

  • k0s ships containerd 2.3.5 with its own config in /etc/k0s/containerd.toml. That file is version = 3 and has one line that matters here:

    imports = ["/etc/k0s/containerd.d/*.toml"]
    

    The directory existed and was empty.

  • The cluster is locked down: Kyverno denies workloads in default, rejects :latest images, and FluxCD owns every manifest.

  • The source guide (Mirantis t0 blog, January 2026) builds runwasi from source with a Rust toolchain on the node. It writes a drop-in for both Wasmtime and WasmEdge. Then it restarts k0scontroller on a single node that is both controller and worker.

Picking a Runtime

The guide installs two shims, Wasmtime and WasmEdge. I only want one.

WasmtimeWasmEdge
Backed byBytecode AllianceCNCF sandbox project
WASI preview 2 and component modelfirst to ship, reference implementationcatching up
Extrasnone I needWASI-NN and LLM plugins, own socket extensions
Fits my plansRust is the main source (wasm32-wasip1 / wasip2)AI already goes through LiteLLM

I picked Wasmtime. To add WasmEdge later, I need one more drop-in file and one more RuntimeClass.

Setup

No Rust on the nodes

runwasi publishes a static musl build of each shim on its releases page, so I do not have to compile anything. The tarball also carries a keyless sigstore signature (a signature tied to the CI identity, with no long-lived key) from the runwasi release workflow. I checked it before I put a binary next to containerd:

shasum -a 256 -c <<< "a9b1215ee670f11414c8fd8e970b52679085929c8ec73a0fe6ed9cf215cf9ba1  containerd-shim-wasmtime-x86_64-linux-musl.tar.gz"
# containerd-shim-wasmtime-x86_64-linux-musl.tar.gz: OK
cosign verify-blob \
  --certificate containerd-shim-wasmtime-v1.pem \
  --signature containerd-shim-wasmtime-v1.sig \
  --certificate-identity-regexp '^https://github.com/containerd/runwasi/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  containerd-shim-wasmtime-v1
# Verified OK

That is containerd-shim-wasmtime v0.6.1: one 31 MB static binary, containerd-shim-wasmtime-v1, copied to /usr/local/bin on each worker.

Rolling it out one worker at a time

On each worker, I copied the shim and the drop-in and restarted k0sworker. Then I waited for the node to be Ready, made sure that nothing on it stopped, and moved to the next worker.

scp containerd-shim-wasmtime-v1 root@<worker>:/usr/local/bin/
scp wasmtime.toml root@<worker>:/etc/k0s/containerd.d/
ssh root@<worker> systemctl restart k0sworker
kubectl wait --for=condition=Ready node/<worker> --timeout=180s
# ready after 6s

Every worker was Ready again in 5 to 6 seconds. Running pods keep running through a containerd restart because each container lives under its own shim process. The only noise in the journal was kubelet failing to look up a few old, already dead containers.

A k0s node has no crictl. To see what containerd actually loaded, I asked containerd itself for its merged config:

/opt/lib/k0s/bin/containerd --config /etc/k0s/containerd.toml config dump | grep wasmtime
# [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.wasmtime]
#   runtime_type = 'io.containerd.wasmtime.v1'

Issue 1: The Guide’s Config Is for containerd 1.x

Symptom

Nothing happens, and that is the problem. The guide’s drop-in looks like this:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmtime]
runtime_type = "io.containerd.wasmtime.v1"

Cause

That is the containerd 1.x CRI plugin path. containerd 2.x split the CRI plugin and moved runtimes to io.containerd.cri.v1.runtime, with config version = 3. If a table uses a plugin name that no longer exists, containerd ignores it and raises no error. So containerd never registers the runtime, and a RuntimeClass that points at wasmtime has no handler behind it.

Fix

Write the drop-in for config version 3:

version = 3

[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.wasmtime]
  runtime_type = "io.containerd.wasmtime.v1"

config dump then shows the runtime, as above.

Issue 2: The Image Pulls, Then the Pod Cannot Start

For the test, I ran the runwasi demo module in a pod with a RuntimeClass that points at the wasmtime handler. Kyverno first refused it twice, both times correctly: no pods in default, and no :latest. So I used a throwaway namespace and pinned wasi-demo-app:0.2.0 by digest.

Symptom

Pulled   Successfully pulled image "ghcr.io/containerd/runwasi/wasi-demo-app:0.2.0@sha256:1a5e..." in 3.679s. Image size: 2265395 bytes.
Failed   Error: failed to create containerd container: parent snapshot sha256:5e2740... does not exist: not found

Kubelet pulls the image. Then containerd cannot find the filesystem that the image must unpack into.

Cause

The manifest was ordinary: one application/vnd.oci.image.layer.v1.tar layer. The image config was not:

curl -sL -H "Authorization: Bearer $TOK" "$REGISTRY/blobs/$CONFIG_DIGEST" | jq -c '{os,architecture,ep:.config.Entrypoint}'
# {"os":"wasip1","architecture":"wasm","ep":["/wasi-demo-app.wasm"]}

The CRI plugin of containerd downloads the content. Then it unpacks the layers into a snapshot (the unpacked filesystem that a container starts from) for one platform. By default, that is the host platform, linux/amd64. A wasip1/wasm image matches nothing. So the content is in the store, but containerd never makes a snapshot, and container create fails on the missing snapshot.

flowchart LR
  K[kubelet PullImage] -->|no runtime handler| C[containerd CRI]
  C --> P{image platform<br/>= linux/amd64?}
  P -->|yes| U[unpack to snapshot] --> S[create container on wasmtime]
  P -->|"no (wasip1/wasm)"| X["content only, no snapshot<br/>parent snapshot does not exist"]

Dead end: tell containerd the platform per runtime

containerd 2.x has a setting for exactly this. Its CRI config source has RuntimePlatforms, which maps a runtime to the platform that its images use.

[plugins."io.containerd.cri.v1.images".runtime_platforms.wasmtime]
  platform = "wasip1/wasm"

config dump showed it loaded. Same error. The mapping only applies when containerd knows which runtime an image pull is for. Kubelet only sends that information with the RuntimeClassInImageCriAPI feature gate. That gate became alpha in Kubernetes 1.29, and it is still alpha and off by default in 1.36. I will not turn on an alpha kubelet gate on every worker for this.

Dead end: pre-pull by hand

k0s ctr images pull --platform wasip1/wasm failed with no unpack platforms defined, because the transfer service only unpacks for the host platform too. The client-side pull, --local, did work, and the pod then ran:

hello from wasmtime on k0s
exiting

That proved the shim and the drop-in work. But pre-pulling images on every node by hand is not a way to deploy anything.

Fix

The Wasmtime shim does not care about the platform label. It looks at the container’s entrypoint, sees a .wasm module, and runs it. So the image only has to be an image that kubelet unpacks: a scratch image with the module in it, built for linux/amd64.

FROM scratch
COPY wasi-demo-app.wasm /wasi-demo-app.wasm
ENTRYPOINT ["/wasi-demo-app.wasm"]
docker build --platform linux/amd64 -t wasi-demo-amd64:0.2.0 .

The image is a 636 KB tarball. I imported it on the worker and ran it through the normal kubelet path:

wasi-amd64   0/1   Completed   0   11s
linux-amd64 wasm image, normal unpack
exiting

Without the alpha gate, the runtime_platforms mapping does nothing, so I removed it. Then I ran the test again: still Completed.

Making It Stick

  • Node side, in the cloud-init snippet that builds every worker VM: the drop-in is a write_files entry, and the shim download is pinned by sha256. I did a dry run of the snippet’s commands on a worker, and it caught one more thing. The tarball’s members are ./containerd-shim-wasmtime-v1, with a leading ./. So tar -x ... containerd-shim-wasmtime-v1 says Not found in archive. With the ./, it produces a binary identical to the installed one.

  • Cluster side, a Flux app with one cluster-scoped object:

    apiVersion: node.k8s.io/v1
    kind: RuntimeClass
    metadata:
      name: wasmtime
    handler: wasmtime
    

    The shim is on every worker, so it needs no scheduling block.

  • The last check ran on a different node through the Flux-owned RuntimeClass: Completed, with the module’s output in the pod log.

A Wasm pod in this cluster now looks like any other pod with one extra line, runtimeClassName: wasmtime. It still goes through the same Kyverno rules (own namespace, pinned image, securityContext, requests and limits).

Lessons

  1. Check which containerd your distribution ships before copying a config snippet. containerd 2.x moved the CRI plugin, and the old path fails silently. containerd config dump shows what containerd actually loaded.
  2. “Successfully pulled” is not “unpacked”. CRI unpacks for one platform. An image built for another platform stays in the content store. The error only shows up at container create, as a missing snapshot.
  3. Build Wasm images as linux/amd64 until kubelet can tell containerd the runtime. That needs RuntimeClassInImageCriAPI, still alpha in 1.36. The shim reads the entrypoint, not the platform label.
  4. Prebuilt and signed beats building on the node. The workers need no Rust toolchain, the snippet pins the sha256, and a sigstore check ties the binary to the runwasi release workflow.

Series

This post is part of Homelab Rebuild:

  1. Part 1: Upgrading Proxmox VE 8 to 9
  2. Part 2: Starting Over When the IaC Is Broken
  3. Part 3: When Tailscale Breaks Your Cilium Gateway
  4. Part 4: Living on the Tailnet
  5. Part 5: Secrets Without Secrets in Git
  6. Part 6: Certificates, DNS and a Private CA
  7. Part 7: Self-Hosted CI and Getting Back to IaC
  8. Part 8: Stateful Workloads on Unequal Hardware
  9. Part 9: Going Public Safely with Cloudflare Tunnel
  10. Part 10: When the Hypervisor Went Quiet
  11. Part 11: Hardening the Cluster in Five Layers
  12. Part 12: Observability with Coroot, and Losing pve-01
  13. Part 13: The Host That Kept Losing Power
  14. Part 14: Observability, Take Two with OpenTelemetry
  15. Part 15: Requests and Limits for Everything
  16. Part 16: Handing the Cluster to FluxCD
  17. Part 17: Making the Guardrails Earn Their Keep
  18. Part 18: When the API Server Outgrew Its VM
  19. Part 19: Logging In to OpenBao with Cloudflare Access and Google
  20. Part 20: Headlamp SSO, or Teaching kube-apiserver to Trust Cloudflare Access
  21. Part 21: pve-02 Loses Power, and Pi-hole Has Nowhere to Go
  22. Part 22: Turning Four Days of Fixes into AI Runbooks
  23. Part 23: microVM Sandboxes on Proxmox
  24. Part 24: Pi-hole Without a Home Host
  25. Part 25: It Was the Router
  26. Part 26: TLS on the Front, Plaintext Behind It
  27. Part 27: One Bad Power Adapter, Two Different Deaths
  28. Part 28: Observability, Take Three: Making It Worth Looking At
  29. Part 29: One Meter for Every Model, LiteLLM on Workers AI
  30. Part 30: Proxmox Joins the SSO, and a Plan Token Loses Admin
  31. Part 31: First Backups, Postgres, OpenBao and etcd to R2
  32. Part 32: Alerts That Reach Me, Even When the Lab Is Dead
  33. Part 33: The Firewall Held, the Tailnet Did Not
  34. Part 34: From Watching to Blocking, and a Record of Who Did What
  35. Part 35: Wazuh Without the 8 GB, Endpoint Security for One Mac and Three Hosts
  36. Part 36: The Fans Were Loud, the CPUs Were Idle
  37. Part 37: A Power Trip, a Search Domain and a Wildcard
  38. Part 38: An Autonomous AI Pentester in a Sealed Box
  39. Part 39: Wasm Pods on k0s, and the Label That Kept Them Packed (this post)

References