K3s — API Surface#

API types#

CLI + Supervisor HTTP API + Embedded Kubernetes API (proxied)

k3s exposes three distinct API surfaces: a rich CLI for operators and administrators, a custom HTTP “supervisor” API used internally by k3s agents to bootstrap and communicate with the server, and the full upstream Kubernetes API server (proxied through the supervisor port). There are no gRPC services or public library packages.

CLI#

Framework#

github.com/urfave/cli/v2 — not Cobra. This is a notable divergence from the Kubernetes ecosystem norm (Cobra + pflag). k3s uses urfave/cli for its CLI layer but supplements it with github.com/spf13/pflag for early argument scanning (data-dir, debug, prefer-bundled-bin) before the CLI library is fully initialized.

Config file preprocessing#

Before urfave/cli parses flags, pkg/configfilearg.MustParse(os.Args) scans for --config/-c, reads the YAML file, and expands its key-value pairs into equivalent CLI flags. This means the CLI surface and YAML config surface are 1:1 — no separate config schema.

Top-level commands#

The k3s binary is a multicall dispatcher. The same binary is invoked as different tool names via symlinks (crictl, kubectl, ctr). As a CLI app, the root commands are:

Subcommand structure#

k3s token (pkg/cli/cmds/token.go):

• create — Create a bootstrap token with TTL, groups, usages
• delete — Delete a token
• generate — Generate a token string without creating it on the server
• list — List tokens (with -o json option)
• rotate — Rotate the server token

k3s etcd-snapshot (pkg/cli/cmds/etcd_snapshot.go):

• save — Trigger immediate snapshot
• delete — Delete named snapshot(s)
• ls (aliases: list, l) — List snapshots (with -o json option)
• prune — Remove snapshots exceeding configured retention count

k3s secrets-encrypt (pkg/cli/cmds/secrets_encrypt.go):

• status — Show current encryption status
• enable — Enable encryption at rest
• disable — Disable encryption at rest
• prepare — Prepare for key rotation
• rotate — Rotate the current encryption key
• reencrypt — Re-encrypt all secrets with the current key
• rotate-keys — Rotate encryption provider keys (calls Kubernetes KMS)

k3s certificate (pkg/cli/cmds/certs.go):

• check — Validate certificate expiry
• rotate — Rotate expiring certificates
• rotate-ca — Rotate CA certificates

Flag patterns#

• Global flags: --debug (also K3S_DEBUG env), --data-dir/-d (also K3S_DATA_DIR env)
• k3s server flags: 100+ flags covering networking, storage, TLS, feature gates, and component enable/disable. All are in the cmds.Server struct (pkg/cli/cmds/server.go).
• k3s agent flags: ~50 flags covering node identity, networking, and containerd config. All in cmds.Agent struct (pkg/cli/cmds/agent.go).
• Environment variable binding: Key flags have EnvVars: []string{version.ProgramUpper + "_KUBECONFIG_OUTPUT"} etc., using the K3S_ prefix convention throughout.
• No automatic env→flag binding framework (no Viper): each flag explicitly lists its env var names.

REST/HTTP — Supervisor API#

k3s runs a custom HTTP server on the supervisor port (default 6443 in single-port mode, configurable via --supervisor-port). This is k3s’s own API — distinct from the Kubernetes API server that also runs on port 6443. The two are multiplexed: the supervisor NotFoundHandler falls through to the embedded Kubernetes API server handler.

Router#

github.com/gorilla/mux wrapped in a thin k3s mux.Router type (pkg/util/mux/). mux.Router adds a .Use(middlewares...) API on top of gorilla’s router, enabling per-router middleware stacks.

Route registration#

All routes are registered in pkg/server/handlers/router.go:NewHandler(). Routes are hard-coded (no annotation or convention-based discovery). The router uses nested NotFoundHandler chaining to implement layered auth: unauthed → agent-authed → node-authed → server-authed → system-privileged.

Middleware chain (layered auth model)#

The supervisor API uses nested routers rather than a linear middleware chain. Each layer handles a different auth level and delegates non-matching paths inward:

router (no auth)
  └── systemAuthed (system:masters RBAC group required)
        └── serverAuthed (k3s:server group required)
              └── nodeAuthed (system:nodes group required)
                    └── authed (k3s:agent, system:nodes, or bootstrap-token group required)
                          └── NotFoundHandler → Kubernetes APIServer

Each layer adds auth.HasRole(control, ...), auth.RequestInfo(), and auth.MaxInFlight() middleware via mux.Use().

The auth.Delegated() middleware (used for spegel/metrics/pprof sub-routers) delegates authentication/authorization to the Kubernetes API server via SubjectAccessReview.

Authentication#

• auth.HasRole() calls control.Runtime.Authenticator.AuthenticateRequest(req), which is a k3s-specific authenticator that validates bearer tokens and client certificates against the cluster’s CA
• auth.IsLocalOrHasRole() short-circuits auth for requests from 127.0.0.1/::1
• auth.Delegated() uses Kubernetes RBAC SubjectAccessReview for components like spegel and pprof

Key endpoints#

All k3s-specific endpoints are under /v1-k3s/ (the prefix uses the program name for rebrandability):

Handler design: Each handler is a function returning http.Handler (closed over config.Control), not a method on a controller struct. Certificate endpoints use a GET/POST dual pattern: GET returns the legacy shared key + cert; POST with a CSR body returns only the signed cert using the CSR’s public key.

Additional per-node HTTP endpoints (agent nodes)#

Agent nodes expose additional HTTP servers registered via the RouterFunc pattern:

gRPC API#

Not present in k3s’s own code. k3s contains no .proto files and registers no gRPC servers. gRPC appears only in test files (pkg/etcd/etcd_linux_test.go) where mock etcd gRPC servers are stood up to test the k3s etcd client integration. The embedded etcd server itself uses gRPC internally, but that is upstream etcd code, not k3s code.

Plugin / Extension system#

k3s has one explicit extension point at the Go level, plus a structural extension mechanism:

StartupHook (Go API)#

Package: pkg/cli/cmds/ (types), pkg/server/server.go (execution)

type StartupHookArgs struct {
    APIServerReady       <-chan struct{}
    KubeConfigSupervisor string
    Skips                map[string]bool
    Disables             map[string]bool
}

type StartupHook func(context.Context, *sync.WaitGroup, StartupHookArgs) error

Hooks are registered in cmds.Server.StartupHooks []StartupHook and are called after the API server is ready but before the server is considered fully started. This is an in-process extension point — only usable by code that imports k3s packages and builds a custom binary (not by external plugins). Hooks are used by integration tests and by downstream distributions (e.g., RKE2) to register additional controllers.

RouterFunc (Go API)#

Package: pkg/spegel/, pkg/metrics/, pkg/profile/

type RouterFunc func(ctx context.Context, nodeConfig *config.Node) (*mux.Router, error)

Components register themselves by setting a package-level Router variable of type RouterFunc. The agent/server startup code calls these during initialization to attach sub-routers to the node’s HTTP listener. This is the pattern used by spegel (embedded registry mirror), metrics, and pprof.

Embedded executor (build-tag extension)#

The Executor interface in pkg/daemons/executor/ (17 methods) is designed as an extension point. The no_embedded_executor build tag allows building k3s without the upstream Kubernetes components, enabling alternative executor implementations. This is a compile-time extension mechanism used by the test suite and potentially by downstream distributions.

Kubernetes add-on controllers (runtime extension)#

k3s bundles two controller-based extension mechanisms that run as Kubernetes controllers:

• HelmChart CRD controller (k3s-io/helm-controller): Watches HelmChart and HelmChartConfig custom resources and renders/applies Helm charts. This is the primary way to extend k3s with additional cluster functionality — users create HelmChart resources rather than running helm install.
• Auto-deploy manifests (pkg/deploy): Files placed in /var/lib/rancher/k3s/server/manifests/ are auto-applied on startup. k3s ships default manifests for coredns, traefik, local-path-provisioner, metrics-server, and ccm.

Library API#

k3s is not designed as a library. There are no public packages intended for import by third parties. The package structure (all under github.com/k3s-io/k3s/pkg/) is technically importable but:

• The cmds.Server.StartupHooks field is the one documented extension point for downstream forks (e.g., RKE2 builds on top of k3s by importing its packages and adding hooks)
• There is no versioning strategy or stability guarantee for any packages
• The Executor interface is the other “intended” seam, but only accessible via build tags and blank imports

API design observations#

1. Rebrandability via version.Program: All k3s-specific HTTP paths, environment variables, RBAC group names, and annotation keys use version.Program (default "k3s") rather than hardcoded strings. This allows downstream distributions like RKE2 to rebrand by changing this constant at build time.

2. Certificate issuance as an API: The supervisor API’s most architecturally interesting feature is its certificate-signing service. Agents authenticate with a join token, then receive signed TLS certificates for all Kubernetes components on-demand. The dual GET/POST pattern (shared key vs. CSR-based) provides a migration path from the legacy shared-key model to a more secure CSR-based model.

3. Layered NotFoundHandler chain: The nested router auth model is unusual. Rather than a flat middleware chain, each auth tier is a separate gorilla/mux router with its NotFoundHandler pointing to the next tier. This makes the auth model explicit in code structure but makes it harder to trace the full middleware chain for any given request.

4. No public REST API versioning: The k3s supervisor API has no version negotiation, content negotiation, or deprecation strategy. It is a private, versioned-by-release API. Only agents built from the same version are expected to talk to it.

5. Uniform flag naming: The k3s server command has 100+ flags but follows consistent naming conventions: --disable-<component> to remove built-in components, --extra-<component>-arg to pass additional flags to embedded Kubernetes components, and K3S_<FLAG_NAME> environment variable equivalents for all major flags.