Back to Home
post-thumb

Homelab Rebuild Part 39: Wasm Pods on k0s, and the Label That Kept Them Packed

| October 26, 2026 | 11 min to read

Intro

I want to run some of my own small services as WebAssembly modules on the k0s cluster, next to the normal containers. A Mirantis blog post walks through exactly that on k0s: build the runwasi shims, drop a containerd config file into /etc/k0s/containerd.d/, add a RuntimeClass, done. So the question was whether the guide holds up on my cluster, and which Wasm runtime to pick.

The guide’s mechanism is right, but two things in it do not work on a current k0s, and the second one cost me most of a morning.

TL;DR: use the prebuilt, signed runwasi Wasmtime shim and a containerd drop-in, but on k0s 1.36 (containerd 2.x) the runtime goes under io.containerd.cri.v1.runtime, not the containerd 1.x path most guides show. And build your Wasm images as linux/amd64, not wasip1/wasm: kubelet pulls with the host platform, so a wasip1/wasm image is pulled but never unpacked, and the pod dies with parent snapshot ... does not exist.

The Figure

containerd 2.3 · config v3
Tonearms: the runtime handler, picked by runtimeClassName (back runc, front wasmtime)The changer: kubelet and CRI pull with the default platform, so only a linux/amd64 record reaches the platterThe card in the slot: the drop-in /etc/k0s/containerd.d/wasmtime.toml; the greyed switch: RuntimeClassInImageCriAPI (alpha, off)Each record is an image and its label is the platform. The changer is kubelet and CRI: they pull with the default platform and unpack only a linux/amd64 image into a snapshot, the platter. The upstream wasi-demo-app is wasip1/wasm, so it pulls but never lands: CreateContainerError, parent snapshot does not exist. Rebuilt as a linux/amd64 scratch image (636 KB) it drops onto the platter. The wasmtime arm (runwasi containerd-shim-wasmtime-v1 v0.6.1, registered by the drop-in card) plays it and the pod prints hello from wasmtime on k0s and reaches Completed. The runc arm tries to run the same .wasm file as a Linux binary: exec format error.

The figure is a jukebox drawn as a record changer. Each record is an image, and its label is the platform. The carriage on the overhead rail is kubelet and CRI: they pull with the default platform, so only a linux/amd64 record gets lifted onto the platter (the unpacked snapshot). A wasip1/wasm record stays in the magazine and the lamp beside it lights, because the image pulled but was never unpacked. Two tonearms stand at the right of the platter: runc at the back and wasmtime in front, chosen by runtimeClassName. The card in the slot is the drop-in wasmtime.toml, and the small greyed switch is RuntimeClassInImageCriAPI, still alpha and off. Press P or click the record to flip the platform, and click an arm (or press R/W) to pick the runtime: wasip1/wasm fails with parent snapshot does not exist on either arm, linux/amd64 on wasmtime prints hello from wasmtime on k0s and reaches Completed, and linux/amd64 on runc gets exec format error. Built with the anatomy skill.

The Situation

  • k0s v1.36.4+k0s.1 on 5 Debian 13 workers and 3 controllers, data dir /opt/lib/k0s. Workers run the k0sworker service.

  • k0s ships containerd 2.3.5 with its own config in /etc/k0s/containerd.toml. That file is version = 3 and has one line that matters here:

    imports = ["/etc/k0s/containerd.d/*.toml"]
    

    The directory existed and was empty.

  • The cluster is locked down: Kyverno denies workloads in default, rejects :latest images, and FluxCD owns every manifest.

  • The source guide (Mirantis t0 blog, January 2026) builds runwasi from source with a Rust toolchain on the node, writes a drop-in for both Wasmtime and WasmEdge, and restarts k0scontroller on a single node that is both controller and worker.

Picking a Runtime

The guide installs two shims, Wasmtime and WasmEdge. I only want one.

WasmtimeWasmEdge
Backed byBytecode AllianceCNCF sandbox project
WASI preview 2 and component modelfirst to ship, reference implementationcatching up
Extrasnone I needWASI-NN and LLM plugins, own socket extensions
Fits my plansRust is the main source (wasm32-wasip1 / wasip2)AI already goes through LiteLLM

Wasmtime it is. Adding WasmEdge later is one more drop-in file and one more RuntimeClass.

Setup

No Rust on the nodes

runwasi publishes a static musl build of each shim on its releases page, so there is nothing to compile. The tarball also carries a keyless sigstore signature from runwasi’s own release workflow, which is worth checking before putting a binary next to containerd:

shasum -a 256 -c <<< "a9b1215ee670f11414c8fd8e970b52679085929c8ec73a0fe6ed9cf215cf9ba1  containerd-shim-wasmtime-x86_64-linux-musl.tar.gz"
# containerd-shim-wasmtime-x86_64-linux-musl.tar.gz: OK
cosign verify-blob \
  --certificate containerd-shim-wasmtime-v1.pem \
  --signature containerd-shim-wasmtime-v1.sig \
  --certificate-identity-regexp '^https://github.com/containerd/runwasi/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  containerd-shim-wasmtime-v1
# Verified OK

That is containerd-shim-wasmtime v0.6.1: one 31 MB static binary, containerd-shim-wasmtime-v1, copied to /usr/local/bin on each worker.

Rolling it out one worker at a time

Per worker: copy the shim and the drop-in, restart k0sworker, wait for the node to be Ready, check nothing on it stopped running, then move on.

scp containerd-shim-wasmtime-v1 root@<worker>:/usr/local/bin/
scp wasmtime.toml root@<worker>:/etc/k0s/containerd.d/
ssh root@<worker> systemctl restart k0sworker
kubectl wait --for=condition=Ready node/<worker> --timeout=180s
# ready after 6s

Every worker was Ready again in 5 to 6 seconds. Running pods keep running through a containerd restart because each container lives under its own shim process. The only noise in the journal was kubelet failing to look up a few old, already dead containers.

There is no crictl on a k0s node, so to see what containerd actually loaded I asked containerd itself for its merged config:

/opt/lib/k0s/bin/containerd --config /etc/k0s/containerd.toml config dump | grep wasmtime
# [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.wasmtime]
#   runtime_type = 'io.containerd.wasmtime.v1'

Issue 1: The Guide’s Config Is for containerd 1.x

Symptom

Nothing, which is the problem. The guide’s drop-in looks like this:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.wasmtime]
runtime_type = "io.containerd.wasmtime.v1"

Cause

That is the containerd 1.x CRI plugin path. containerd 2.x split the CRI plugin and moved runtimes to io.containerd.cri.v1.runtime, with config version = 3. A table under a plugin name that no longer exists is not an error, it is just ignored, so the runtime is never registered and a RuntimeClass pointing at wasmtime has no handler behind it.

Fix

Write the drop-in for config version 3:

version = 3

[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.wasmtime]
  runtime_type = "io.containerd.wasmtime.v1"

config dump then shows the runtime, as above.

Issue 2: The Image Pulls, Then the Pod Cannot Start

The test is runwasi’s own demo module in a pod with a RuntimeClass pointing at the wasmtime handler. Kyverno first refused it twice, both times correctly: no pods in default, and no :latest. So I used a throwaway namespace and pinned wasi-demo-app:0.2.0 by digest.

Symptom

Pulled   Successfully pulled image "ghcr.io/containerd/runwasi/wasi-demo-app:0.2.0@sha256:1a5e..." in 3.679s. Image size: 2265395 bytes.
Failed   Error: failed to create containerd container: parent snapshot sha256:5e2740... does not exist: not found

The image is pulled, and then containerd cannot find the filesystem the image is supposed to unpack into.

Cause

The manifest was ordinary: one application/vnd.oci.image.layer.v1.tar layer. The image config was not:

curl -sL -H "Authorization: Bearer $TOK" "$REGISTRY/blobs/$CONFIG_DIGEST" | jq -c '{os,architecture,ep:.config.Entrypoint}'
# {"os":"wasip1","architecture":"wasm","ep":["/wasi-demo-app.wasm"]}

containerd’s CRI plugin downloads the content and then unpacks the layers into a snapshot for one platform, by default the host’s, linux/amd64. A wasip1/wasm image matches nothing, so the content is in the store but no snapshot is ever made, and container create fails on the missing snapshot.

flowchart LR
  K[kubelet PullImage] -->|no runtime handler| C[containerd CRI]
  C --> P{image platform<br/>= linux/amd64?}
  P -->|yes| U[unpack to snapshot] --> S[create container on wasmtime]
  P -->|"no (wasip1/wasm)"| X["content only, no snapshot<br/>parent snapshot does not exist"]

Dead end: tell containerd the platform per runtime

containerd 2.x has exactly the knob for this. In its CRI config source there is RuntimePlatforms: map a runtime to the platform its images use.

[plugins."io.containerd.cri.v1.images".runtime_platforms.wasmtime]
  platform = "wasip1/wasm"

config dump showed it loaded. Same error. The mapping only applies when containerd knows which runtime an image pull is for, and kubelet only sends that with the RuntimeClassInImageCriAPI feature gate. That gate has been alpha since Kubernetes 1.29 and is still alpha and off by default in 1.36. I am not turning on an alpha kubelet gate on every worker for this.

Dead end: pre-pull by hand

k0s ctr images pull --platform wasip1/wasm failed with no unpack platforms defined, because the transfer service only unpacks for the host platform too. The client-side pull, --local, did work, and the pod then ran:

hello from wasmtime on k0s
exiting

That proved the shim and the drop-in work. But pre-pulling images on every node by hand is not a way to deploy anything.

Fix

The Wasmtime shim does not care about the platform label. It looks at the container’s entrypoint, sees a .wasm module, and runs it. So the image only has to be something kubelet will unpack: a scratch image with the module in it, built for linux/amd64.

FROM scratch
COPY wasi-demo-app.wasm /wasi-demo-app.wasm
ENTRYPOINT ["/wasi-demo-app.wasm"]
docker build --platform linux/amd64 -t wasi-demo-amd64:0.2.0 .

The image is a 636 KB tarball. Imported on the worker and run through the normal kubelet path:

wasi-amd64   0/1   Completed   0   11s
linux-amd64 wasm image, normal unpack
exiting

Then I took the runtime_platforms mapping back out, since without the alpha gate it does nothing, and ran the test again: still Completed.

Making It Stick

  • Node side, in the cloud-init snippet every worker VM is built from: the drop-in as a write_files entry, and the shim download pinned by sha256. Dry-running the snippet’s commands on a worker caught one more thing: the tarball’s members are ./containerd-shim-wasmtime-v1, with a leading ./, so tar -x ... containerd-shim-wasmtime-v1 says Not found in archive. With the ./ it produces a binary identical to the installed one.

  • Cluster side, a Flux app with one cluster-scoped object:

    apiVersion: node.k8s.io/v1
    kind: RuntimeClass
    metadata:
      name: wasmtime
    handler: wasmtime
    

    The shim is on every worker, so it needs no scheduling block.

  • The last check ran on a different node through the Flux-owned RuntimeClass: Completed, with the module’s output in the pod log.

A Wasm pod in this cluster now looks like any other pod with one extra line, runtimeClassName: wasmtime, and it still goes through the same Kyverno rules (own namespace, pinned image, securityContext, requests and limits).

Lessons

  1. Check which containerd your distribution ships before copying a config snippet. containerd 2.x moved the CRI plugin, and the old path fails silently. containerd config dump is the only honest answer.
  2. “Successfully pulled” is not “unpacked”. CRI unpacks for one platform. An image built for another platform sits in the content store and the error only shows up at container create, as a missing snapshot.
  3. Build Wasm images as linux/amd64 until kubelet can tell containerd the runtime. That needs RuntimeClassInImageCriAPI, still alpha in 1.36. The shim reads the entrypoint, not the platform label.
  4. Prebuilt and signed beats building on the node. No Rust toolchain on the workers, a sha256 pin in the snippet, and a sigstore check that ties the binary to runwasi’s release workflow.

Series

This post is part of Homelab Rebuild:

  1. Part 1: Upgrading Proxmox VE 8 to 9
  2. Part 2: Starting Over When the IaC Is Broken
  3. Part 3: When Tailscale Breaks Your Cilium Gateway
  4. Part 4: Living on the Tailnet
  5. Part 5: Secrets Without Secrets in Git
  6. Part 6: Certificates, DNS and a Private CA
  7. Part 7: Self-Hosted CI and Getting Back to IaC
  8. Part 8: Stateful Workloads on Unequal Hardware
  9. Part 9: Going Public Safely with Cloudflare Tunnel
  10. Part 10: When the Hypervisor Went Quiet
  11. Part 11: Hardening the Cluster in Five Layers
  12. Part 12: Observability with Coroot, and Losing pve-01
  13. Part 13: The Host That Kept Losing Power
  14. Part 14: Observability, Take Two with OpenTelemetry
  15. Part 15: Requests and Limits for Everything
  16. Part 16: Handing the Cluster to FluxCD
  17. Part 17: Making the Guardrails Earn Their Keep
  18. Part 18: When the API Server Outgrew Its VM
  19. Part 19: Logging In to OpenBao with Cloudflare Access and Google
  20. Part 20: Headlamp SSO, or Teaching kube-apiserver to Trust Cloudflare Access
  21. Part 21: pve-02 Loses Power, and Pi-hole Has Nowhere to Go
  22. Part 22: Turning Four Days of Fixes into AI Runbooks
  23. Part 23: microVM Sandboxes on Proxmox
  24. Part 24: Pi-hole Without a Home Host
  25. Part 25: It Was the Router
  26. Part 26: TLS on the Front, Plaintext Behind It
  27. Part 27: One Bad Power Adapter, Two Different Deaths
  28. Part 28: Observability, Take Three: Making It Worth Looking At
  29. Part 29: One Meter for Every Model, LiteLLM on Workers AI
  30. Part 30: Proxmox Joins the SSO, and a Plan Token Loses Admin
  31. Part 31: First Backups, Postgres, OpenBao and etcd to R2
  32. Part 32: Alerts That Reach Me, Even When the Lab Is Dead
  33. Part 33: The Firewall Held, the Tailnet Did Not
  34. Part 34: From Watching to Blocking, and a Record of Who Did What
  35. Part 35: Wazuh Without the 8 GB, Endpoint Security for One Mac and Three Hosts
  36. Part 36: The Fans Were Loud, the CPUs Were Idle
  37. Part 37: A Power Trip, a Search Domain and a Wildcard
  38. Part 38: An Autonomous AI Pentester in a Sealed Box
  39. Part 39: Wasm Pods on k0s, and the Label That Kept Them Packed (this post)

References