Moby — Patterns#

Sampling strategy#

Moby is an XL project (~5,000+ Go files). Pattern detection used grep-based surveys across the entire codebase followed by deep reads of the most illustrative files. Key files examined: errdefs/defs.go, errdefs/helpers.go, daemon/events/events.go, daemon/cluster/cluster.go, client/client_options.go, daemon/list.go, daemon/daemon.go, daemon/server/middleware/middleware.go, daemon/command/daemon.go, daemon/stats/collector.go, pkg/plugins/client.go, daemon/command/trap/trap.go. Architecture and interfaces results were read first.

Concurrency patterns#

Worker pool via errgroup.SetLimit#

• Usage: At least 4 sites in the daemon: daemon/list.go, daemon/server/router/system/system_routes.go, daemon/internal/builder-next/builder.go, daemon/internal/builder-next/adapters/snapshot/layer.go.
• Example: daemon/list.go:135–160 — containers listing uses log2(numContainers) workers:

  numWorkers := max(int(math.Log2(float64(numContainers))), 1)
  g, ctx := errgroup.WithContext(ctx)
  g.SetLimit(numWorkers)
  for i := range containerList {
      g.Go(func() error { ... })
  }
  return g.Wait()

• Assessment: Excellent pattern — bounded concurrency prevents goroutine explosions under high container counts. The log2(N) heuristic is self-documenting in the comment. errgroup provides automatic error collection and context cancellation on first failure. This is idiomatic Go 1.21+ usage.

Pub/sub channel-based event bus#

• Usage: daemon/events/events.go — the primary daemon event stream. Also daemon/stats/collector.go for per-container stats fan-out. Uses the external github.com/moby/pubsub library.
• Example:

  type Events struct {
      mu     sync.Mutex
      events []eventtypes.Message  // circular ring buffer of last 256
      pub    *pubsub.Publisher
  }
  func (e *Events) Subscribe() ([]eventtypes.Message, chan any, func()) {
      l := e.pub.Subscribe()
      cancel := func() { e.Evict(l) }
      return buffered, l, cancel
  }

• Assessment: Clean pattern — returns buffered history + live channel + cancel func in a single call, preventing race between “give me past events” and “subscribe to future events” with a single mutex hold. The chan any type requires type assertion by consumers; this predates Go generics but is a reasonable choice given the age of the codebase.

Two-mutex pattern for long-running reconfiguration#

• Usage: daemon/cluster/cluster.go — exclusively for Swarm cluster state management. Uniquely, the code has a detailed comment block explaining the locking discipline (lines 1–37 of cluster.go).
• Pattern:

  type Cluster struct {
      controlMutex sync.Mutex    // held for full lifecycle of init/join/leave
      mu           sync.RWMutex  // held for state reads during reconfiguration
      nr           *NodeRunner
      ...
  }

  • controlMutex: prevents concurrent reconfiguration operations (join + leave simultaneously). Held for the entire duration of slow operations.
  • mu (RWMutex): allows reads (e.g., network stack asking for current state) to proceed even while controlMutex is held by a long-running operation.
• Assessment: Sophisticated and well-documented. The two-mutex pattern is uncommon in typical Go code but appropriate here: it separates “I am doing a big operation” from “I am reading current state.” The code comment explicitly teaches the pattern to future maintainers. Worth highlighting as a book example of mutex design for operational reliability.

Graceful shutdown via signal channel + WaitGroup#

• Usage: daemon/command/daemon.go:363–398 and daemon/command/trap/trap.go.
• Pattern:

  c := make(chan os.Signal, forceQuitCount)
  signal.Notify(c, os.Interrupt, syscall.SIGTERM)
  // ...
  var apiWG, apiStartWG sync.WaitGroup
  apiStartWG.Wait()  // block until all listeners are ready
  // ... on signal ...
  apiWG.Wait()       // drain in-flight requests

  Second signal (while already shutting down) triggers os.Exit(128 + signum). SIGHUP wires into a separate goroutine that calls daemon.Reload().
• Assessment: Standard Go shutdown idiom, well-executed. The two-WaitGroup approach (apiStartWG for readiness, apiWG for drain) is a clean separation that prevents the daemon from advertising readiness before listeners are open.

sync.Cond for stats polling#

• Usage: daemon/stats/collector.go — the stats collector uses sync.Cond to wake workers when new containers are added or removed.
• Pattern:

  type Collector struct {
      m    sync.Mutex
      cond *sync.Cond
      publishers map[*container.Container]*pubsub.Publisher
  }
  s.cond = sync.NewCond(&s.m)

• Assessment: One of the few legitimate sync.Cond usages in modern Go code. Here it is appropriate: the stats polling loop needs to sleep until there are containers to poll, and sync.Cond.Broadcast() wakes it when the container set changes. A channel-based approach would have been equally valid but the Cond is idiomatic for this “wake on set change” pattern.

Context cancellation — pervasive#

• Usage: 2,267 context.Context function parameter occurrences across the non-vendor codebase. 193 context.WithCancel/WithTimeout/WithDeadline call sites.
• Pattern: Context is threaded through virtually every public function, from HTTP handlers down to containerd gRPC calls. Context keys use unexported struct types to avoid collisions:

  type APIVersionKey struct{}       // daemon/server/httputils
  type UAStringKey struct{}         // dockerversion

• Assessment: Exemplary context discipline. No string keys for context values (uses struct types instead). Context cancellation from the HTTP request is propagated all the way to the containerd gRPC call, enabling proper cleanup if a client disconnects mid-operation.

Rate limiting via golang.org/x/time/rate#

• Usage: 4 sites — client/pkg/progress/progressreader.go, daemon/internal/progress/progressreader.go, daemon/internal/builder-next/adapters/containerimage/pull.go, daemon/logger/logger_error.go, daemon/libnetwork/resolver.go.
• Example: daemon/logger/logger_error.go — rate-limits error log spam from failing logging drivers; daemon/libnetwork/resolver.go — rate-limits DNS responses to prevent amplification attacks.
• Assessment: Precise, appropriate use of token bucket rate limiting. The DNS rate limiting is a security-aware choice, not just performance.

reexec — process re-execution for privilege operations#

• Usage: cmd/dockerd/main.go:17 (reexec.Init()), with registrations in daemon/graphdriver/windows/windows.go, daemon/libnetwork/sandbox_externalkey_unix.go, daemon/builder/dockerfile/copy_windows.go.
• Pattern: The daemon binary registers handlers with reexec.Register(name, fn) at init. When a privileged operation is needed (e.g., setting a network namespace key, writing a Windows layer), the daemon re-executes itself with a specific argument. reexec.Init() at startup checks if the binary was invoked this way and dispatches to the handler, then exits. This avoids fork(2) without exec(2), which is unsafe with goroutines.
• Assessment: A Moby-specific, Go-idiomatic solution to the “fork without exec is unsafe” problem. It is elegant but requires registering handlers before reexec.Init() is called. Worth understanding for anyone building daemons that manage Linux namespaces.

Error handling#

• Style: Mixed — legacy github.com/pkg/errors (240 import sites) coexists with stdlib errors.Is/errors.As (438 call sites). Active migration from the former to the latter is visible in code comments and recent commits.
• Dominant approach: Custom marker interface taxonomy via the errdefs package — unique to Moby and architecturally significant.

errdefs marker interface pattern#

// errdefs/defs.go — public interfaces (zero methods each, except one marker method)
type ErrNotFound interface { NotFound() }
type ErrConflict interface { Conflict() }
type ErrInvalidParameter interface { InvalidParameter() }
type ErrUnauthorized interface { Unauthorized() }
// ... 12 total categories

// errdefs/helpers.go — private wrapper types
type errNotFound struct{ error }
func (errNotFound) NotFound() {}
func (e errNotFound) Unwrap() error { return e.error }

// Constructor — idempotent (returns as-is if already the right type)
func NotFound(err error) error {
    if err == nil || cerrdefs.IsNotFound(err) { return err }
    return errNotFound{err}
}

// Classification — used in HTTP handler to map to status codes
func IsNotFound(err error) bool { var e ErrNotFound; return errors.As(err, &e) }

• Error types defined: 12 semantic categories: ErrNotFound, ErrConflict, ErrInvalidParameter, ErrUnauthorized, ErrForbidden, ErrUnavailable, ErrSystem, ErrNotModified, ErrNotImplemented, ErrUnknown, ErrCancelled, ErrDeadline.
• HTTP mapping: daemon/server/httputils/errors.go uses errdefs.IsNotFound() → 404, errdefs.IsConflict() → 409, etc. The API layer never inspects error strings.
• Wrapping approach: fmt.Errorf("%w", ...) for new code; errors.WithStack() from github.com/pkg/errors in legacy code. The errNotFound wrapper implements both Cause() (pkg/errors compatibility) and Unwrap() (stdlib compatibility).
• Assessment: Sophisticated and well-designed. The marker interface approach is more flexible than sentinel errors (wrapping preserves the original message) and more composable than custom types (any error can be classified). The dual Cause()/Unwrap() implementation shows careful backward-compatibility thinking during the migration. Book-worthy pattern.

errConnectionFailed — struct-wrapping for typed errors#

• File: client/request.go — wraps errors with connection failure context:

  type errConnectionFailed struct{ error }
  func (e errConnectionFailed) Error() string { return ... }

• Assessment: Simpler variant of the errdefs pattern for a specific error class in the client library.

Configuration pattern#

• Approach: Functional options in client/ package; flat config struct + mergo in the daemon.

Functional options in client/#

type clientConfig struct { /* ~12 fields */ }
type Opt func(*clientConfig) error   // the option type

// 15 named constructors:
func WithHost(host string) Opt { ... }
func WithTLSClientConfig(ca, cert, key string) Opt { ... }
func WithDialContext(fn func(ctx, network, addr string) (net.Conn, error)) Opt { ... }
func WithAPIVersion(version string) Opt { ... }
// ...

// Applied in constructor:
func NewClientWithOpts(ops ...Opt) (*Client, error) {
    for _, op := range ops {
        if err := op(&c.clientConfig); err != nil { return nil, err }
    }
}

• Naming: Uses Opt (not Option) as the type name — a minor stylistic choice.
• Error-returning options: Options return error, allowing validation during construction. This is stricter than the typical func(*T) pattern but appropriate for a public library where misconfiguration should fail fast.
• Assessment: Textbook functional options, well-executed. The error-returning variant is appropriate for library code where client misconfiguration is a programming error.

Daemon config — flag struct + mergo merge + atomic.Pointer hot reload#

// daemon.Daemon
configStore atomic.Pointer[configStore]   // daemon/daemon.go:106

// Hot reload on SIGHUP:
func (daemon *Daemon) Reload(conf *config.Config) error {
    newStore := *daemon.configStore.Load()
    newStore.config = *conf
    daemon.configStore.Store(&newStore)   // atomic swap
}

• Assessment: atomic.Pointer[T] is the canonical Go 1.19+ pattern for a hot-swappable config. Readers take a snapshot with Load() and operate on the snapshot; writers atomically replace the pointer. No lock required for readers. The one-field-per-flag pattern in config.Config (~70 fields) is verbose but straightforward.

Dependency injection#

• Approach: Manual constructor wiring — no DI framework.
• Evidence: daemon/command/daemon.go (daemonCLI.start()) — ~400-line initialization sequence that explicitly constructs each subsystem and passes dependencies as parameters. The daemon.Daemon god-struct acts as the ambient context; subsystems that need access to other subsystems receive them as constructor parameters or interface arguments.
• Pattern for subsystem isolation: Rather than passing *daemon.Daemon to HTTP route handlers, each handler package defines a Backend interface containing only the methods it needs:

  // daemon/server/router/container/backend.go
  type Backend interface {
      ContainerCreate(ctx, config, hostCfg, networkCfg, platform, name) (string, error)
      ContainerStart(ctx, name, checkpoint, checkpointDir string) error
      // ~30 methods specific to container operations
  }

  *daemon.Daemon satisfies this interface via its method set. The injection is done in buildRouters().
• Assessment: The Backend-per-router pattern is the most interesting DI detail here — it enforces ISP without a DI framework. The broad Daemon god-struct is the pragmatic trade-off for a single-process system that needs all subsystems simultaneously.

Other notable patterns#

Functional options#

As described above, used pervasively in client/. Also used in BuildKit integration (daemon/internal/builder-next) where the buildkit.Opt struct configures the builder worker.

Table-driven patterns#

• Prevalence: Very heavy — 594 occurrences of testCases/testcases/tc.name/tt.name patterns in test files.
• Style: Anonymous struct slices with named fields. Standard Go idiom.
• Example: client/*_test.go — virtually every client method has a table-driven test.

Interface embedding#

• Prevalence: Moderate. Key examples: client.APIClient is a composite of ~20 sub-interfaces embedded together for testability. libcontainerd types embed nothing but the Backend interface hierarchy uses composition.
• Notable: daemon/container/store.go — Store embeds memdb.MemDB-backed lookup, presented as an interface.

Type assertions / type switches#

• Count: 53 type switch statements.
• Primary use: Error type classification before the errdefs taxonomy was fully adopted (legacy code), and in the event system where chan any carries heterogeneous event types.

Generics usage (Go 1.18+)#

• Limited but targeted: client/utils.go:117 — decodeWithRaw[T any] eliminates repetitive decode + return-raw-JSON boilerplate across the 60+ client API methods.
• Assessment: Judicious use — not pervasive, applied where it eliminates real duplication. Suggests the project adopts generics conservatively, which is appropriate for a widely-deployed library.

Registry pattern#

• Usage: Layer store (layerStore.Register()), volume drivers (store.Register(d, name)), BuildKit source manager (sm.Register(source)), plugin store (reexec.Register()).
• Pattern: Named registration at init time; lookup by name or ID at runtime. The plugin system uses both reexec.Register() (for in-process worker handlers) and a runtime plugin store (for external v2 plugins).

Plugin HTTP-RPC pattern#

• Usage: pkg/plugins/client.go — external plugins (authorization, logging, network drivers, volume drivers) communicate over a Unix socket using a JSON-over-HTTP protocol.
• Pattern: The daemon acts as an HTTP client calling POST /Plugin.Method on the plugin’s socket. Plugin lifecycle is managed by pkg/plugins/plugins.go with retry/backoff logic.
• Assessment: Simple and debuggable. Plugin developers can implement the protocol in any language. The trade-off is higher overhead than gRPC and no schema enforcement, but for plugins that are rarely called (authz, volume operations), this is acceptable.

sync.OnceValue / sync.OnceValues (Go 1.21)#

• Usage: 8 sites — client/client.go:120 (default user agent string), daemon/daemon.go:826 (system check), daemon/pkg/oci/defaults.go:193 (masked paths list), daemon/internal/platform/platform_linux.go:18 (CPU list), daemon/libnetwork/ns/init_linux.go:21 (namespace handles).
• Assessment: Good adoption of Go 1.21’s sync.OnceValue for lazy initialization of expensive/fallible computations. The project is actively modernizing; these usages replaced sync.Once + manual error handling.

Backoff/retry#

• Usage: pkg/plugins/client.go:171 (callWithRetry) and pkg/plugins/plugins.go:200 (loadWithRetry) — exponential backoff for plugin socket connection.
• Pattern: backoff(retries int) time.Duration returns time.Duration(math.Pow(2, float64(retries))) * 100ms, capped at 2 seconds.
• Assessment: Simple exponential backoff without jitter — adequate for the plugin use case (local Unix sockets) but would be problematic for distributed systems. The moby/swarmkit side uses more sophisticated backoff.

stdcopy — custom stream multiplexing#

• Usage: api/pkg/stdcopy/stdcopy.go — a custom binary protocol for multiplexing stdout/stderr over a single TCP/Unix connection without a TTY.
• Protocol: 8-byte header per frame: 1 byte stream type (stdin=0, stdout=1, stderr=2), 3 bytes padding, 4 bytes big-endian payload size.
• Assessment: Pre-dates widespread adoption of gRPC stream multiplexing. It solves a real problem (TTY mode vs raw mode) but is Moby-specific protocol knowledge that consumers must handle. The StdCopy() function handles the demultiplexing.

Pattern summary for the book#