Nomad — Interfaces#

Interface catalog#

Scheduler#

  • Package: scheduler/structs
  • File: scheduler/structs/interfaces.go:30
  • Methods:
    Process(*structs.Evaluation) error
  • Purpose: The single-method contract for all scheduling algorithms. A Scheduler receives one evaluation at a time, reads global state via the State interface, and submits allocation proposals via the Planner interface. Deliberately minimal — business logic lives in the implementation, not the contract.
  • Implementations: GenericScheduler (service/batch jobs), SystemScheduler (system jobs), SysBatchScheduler (system batch jobs), CoreScheduler (internal GC tasks) — all in scheduler/.
  • Design quality: Excellent ISP compliance. One method, one purpose. The factory function Factory func(log.Logger, chan<- interface{}, State, Planner) Scheduler captures the full construction contract, including dependencies, in one type alias.

State (scheduler view)#

  • Package: scheduler/structs
  • File: scheduler/structs/interfaces.go:41
  • Methods:
    Config() *state.StateStoreConfig
Nodes(ws memdb.WatchSet) (memdb.ResultIterator, error)
NodesByNodePool(ws memdb.WatchSet, poolName string) (memdb.ResultIterator, error)
NodePoolByName(ws memdb.WatchSet, poolName string) (*structs.NodePool, error)
AllocsByJob(ws memdb.WatchSet, namespace, jobID string, all bool) ([]*structs.Allocation, error)
AllocsByNode(ws memdb.WatchSet, node string) ([]*structs.Allocation, error)
AllocByID(ws memdb.WatchSet, allocID string) (*structs.Allocation, error)
AllocsByNodeTerminal(ws memdb.WatchSet, node string, terminal bool) ([]*structs.Allocation, error)
NodeByID(ws memdb.WatchSet, nodeID string) (*structs.Node, error)
JobByID(ws memdb.WatchSet, namespace, id string) (*structs.Job, error)
DeploymentsByJobID(ws memdb.WatchSet, namespace, jobID string, all bool) ([]*structs.Deployment, error)
JobByIDAndVersion(ws memdb.WatchSet, namespace, id string, version uint64) (*structs.Job, error)
LatestDeploymentByJobID(ws memdb.WatchSet, namespace, jobID string) (*structs.Deployment, error)
SchedulerConfig() (uint64, *structs.SchedulerConfiguration, error)
CSIVolumeByID(memdb.WatchSet, string, string) (*structs.CSIVolume, error)
CSIVolumesByNodeID(memdb.WatchSet, string, string) (memdb.ResultIterator, error)
HostVolumeByID(memdb.WatchSet, string, string, bool) (*structs.HostVolume, error)
HostVolumesByNodeID(memdb.WatchSet, string, state.SortOption) (memdb.ResultIterator, error)
TaskGroupHostVolumeClaimsByFields(memdb.WatchSet, state.TgvcSearchableFields) (memdb.ResultIterator, error)
LatestIndex() (uint64, error)
  • Purpose: A read-only view of the global cluster state. The scheduler package has zero import dependency on the nomad/ server package — this interface is the firewall. All queries return go-memdb iterators or typed slices; watch sets allow blocking queries. The actual implementation is nomad/state.StateStore, passed to workers as this narrower interface.
  • Implementations: nomad/state.StateStore (production), plus test stubs in scheduler/ test files.
  • Design quality: Broader than most Go interfaces (20+ methods), but each method maps directly to a query the scheduler genuinely needs. The watch-set pattern (memdb.WatchSet parameter on every read) is a deliberate design allowing callers to subscribe to future changes — integral to Nomad’s blocking query model.

Planner#

  • Package: scheduler/structs
  • File: scheduler/structs/interfaces.go:108
  • Methods:
    SubmitPlan(*structs.Plan) (*structs.PlanResult, State, error)
UpdateEval(*structs.Evaluation) error
CreateEval(*structs.Evaluation) error
ReblockEval(*structs.Evaluation) error
ServersMeetMinimumVersion(minVersion *version.Version, checkFailedServers bool) bool
  • Purpose: Write-only contract for scheduler output. A scheduler submits its proposed allocation plan (Plan) and receives back a PlanResult (what the leader accepted) plus a refreshed State snapshot. This asymmetry — read from State, write via Planner — creates a clean CQRS-style separation between reads and writes within the scheduler.
  • Implementations: nomad.planner struct (embedded in nomad.Server, defined in nomad/plan_endpoint.go).
  • Design quality: Well-segregated. The SubmitPlan return of a fresh State is architecturally clever — after a plan is submitted, the scheduler’s cached state view may be stale, so the refreshed view is returned in-band rather than requiring a separate call.

DriverPlugin#

  • Package: plugins/drivers
  • File: plugins/drivers/driver.go:51
  • Methods:
    // Embedded: base.BasePlugin
PluginInfo() (*PluginInfoResponse, error)
ConfigSchema() (*hclspec.Spec, error)
SetConfig(c *Config) error

// Driver-specific:
TaskConfigSchema() (*hclspec.Spec, error)
Capabilities() (*Capabilities, error)
Fingerprint(context.Context) (<-chan *Fingerprint, error)
RecoverTask(*TaskHandle) error
StartTask(*TaskConfig) (*TaskHandle, *DriverNetwork, error)
WaitTask(ctx context.Context, taskID string) (<-chan *ExitResult, error)
StopTask(taskID string, timeout time.Duration, signal string) error
DestroyTask(taskID string, force bool) error
InspectTask(taskID string) (*TaskStatus, error)
TaskStats(ctx context.Context, taskID string, interval time.Duration) (<-chan *cstructs.TaskResourceUsage, error)
TaskEvents(context.Context) (<-chan *TaskEvent, error)
SignalTask(taskID string, signal string) error
ExecTask(taskID string, cmd []string, timeout time.Duration) (*ExecTaskResult, error)
  • Purpose: The complete lifecycle contract for a task driver. A driver must implement: fingerprinting (capability reporting), task lifecycle (start/wait/stop/destroy), introspection (inspect, stats, events), and optional exec. The interface is also implemented by a generated gRPC proxy client — meaning both in-process drivers and out-of-process (go-plugin) drivers satisfy the same interface, making the dispatch transparent.
  • Implementations: drivers/docker (Docker), drivers/exec (OS process with isolation), drivers/rawexec (raw exec, no isolation), drivers/java (JVM), drivers/qemu (QEMU VMs). Third-party drivers ship as separate binaries.
  • Design quality: Larger interface (15 methods) but correctly sized for a full lifecycle contract. Optional capabilities (SignalTask, ExecTask) are covered by DriverSignalTaskNotSupported and DriverExecTaskNotSupported embed-structs that satisfy those methods with “not supported” errors — a pragmatic escape hatch that avoids splitting the interface further. Additional opt-in interfaces (ExecTaskStreamingDriver, DriverNetworkManager) extend via separate smaller interfaces.

StateDB#

  • Package: client/state
  • File: client/state/interface.go:18
  • Methods: 30+ methods covering Put/Get for allocations, task state, deployment status, network status, plugin state, check results, node metadata, workload identities, host volumes, node identity.
  • Purpose: The client’s persistence contract. The nomad client node stores all live state (running allocations, task state, plugin registrations) in a local BoltDB, but all calls go through this interface, allowing test doubles and future storage backends. Includes BatchMode write option for coalescing concurrent writes into a single BoltDB transaction.
  • Implementations: client/state.BoltStateDB (production BoltDB), client/state.MemDB (in-memory, used in tests and ACL bootstrap).
  • Design quality: Large (30+ methods) but each method is a fine-grained key-value operation with clear symmetry (Put/Get/Delete pairs). The WriteOption functional option pattern for BatchMode is a well-applied extension point that avoids method proliferation. Could be split into multiple narrower interfaces by subsystem (alloc state, plugin state, check results) but the single interface matches the single-backend deployment model.

FeasibleIterator and RankIterator#

  • Package: scheduler/feasible
  • File: scheduler/feasible/feasible.go:60 and scheduler/feasible/rank.go:78
  • Methods:
    // FeasibleIterator
Next() *structs.Node
Reset()

// RankIterator
Next() *RankedNode
Reset()
  • Purpose: The scheduling pipeline is implemented as a composable chain of iterators. FeasibleIterator yields nodes that pass hard constraints (resource availability, driver presence, topology). RankIterator wraps a FeasibleIterator and produces ranked nodes (RankedNode with FinalScore). Multiple iterators can be stacked: StaticIterator → ConstraintIterator → DriverIterator → BinpackIterator. The Reset() method allows the same chain to be replayed after each allocation placement within a single evaluation.
  • Implementations: StaticIterator, RandomIterator, ConstraintChecker, DriverIterator, CSIVolumeIterator, BinpackIterator, MaxScoreIterator, many more — all in scheduler/feasible/.
  • Design quality: Textbook iterator composition pattern, analogous to Go’s io.Reader chains. Two-method interfaces (one per iterator style) are minimal and composable. The ContextualIterator interface (SetJob, SetTaskGroup) handles the parameterization concern without polluting the core iterator contract.

admissionController / jobMutator / jobValidator#

  • Package: nomad (server)
  • File: nomad/job_endpoint_hooks.go:164
  • Methods:
    // admissionController (base)
Name() string

// jobMutator (extends admissionController)
Mutate(*structs.Job) (out *structs.Job, warnings []error, err error)

// jobValidator (extends admissionController)
Validate(*structs.Job) (warnings []error, err error)
  • Purpose: Admission control chain applied to every job submitted to the server. Mutators run first (normalizing, injecting implicit constraints, expanding templates), validators run after (checking semantic correctness). Each hook is a small focused struct with a single responsibility, registered in ordered slices. Both return []error warnings in addition to a hard error, allowing non-fatal issues to surface without blocking the job.
  • Implementations: Nomad internal: jobImplicitConstraintMutator, jobConnectHook, jobNamespacedAttribute, and several validators checking job spec invariants. Enterprise editions add additional mutators.
  • Design quality: Clean, extensible chain-of-responsibility. The separation of jobMutator (transforms) vs jobValidator (asserts) is idiomatic and reduces the risk of a validator inadvertently modifying state.

PluginManager#

  • Package: client/pluginmanager
  • File: client/pluginmanager/manager.go:9
  • Methods:
    Run()
Shutdown()
PluginType() string
  • Purpose: Lifecycle contract for a class of plugins (drivers, devices, CSI). The client holds a slice of PluginManager instances and calls Run() / Shutdown() on each. FingerprintingPluginManager extends this with WaitForFirstFingerprint(context.Context) <-chan struct{}, allowing the client to block node registration until all plugins have reported capabilities.
  • Implementations: drivermanager.Manager, devicemanager.Manager, csimanager.Manager.
  • Design quality: Intentionally minimal. 3-method interface covers start/stop/identity. The fingerprinting extension via interface embedding is a clean optional capability pattern.

Interface patterns#

  • Size distribution: Heavily bimodal. Boundary-isolation interfaces (Scheduler: 1 method; FeasibleIterator: 2; PluginManager: 3; admissionController: 1) are tiny. Persistence and extension-point interfaces (StateDB: 30+; DriverPlugin: 15; State: 20+) are large. The large interfaces model complete subsystem contracts, not individual operations.
  • Embedding: Widely used for optional capability extension. DriverPlugin embeds base.BasePlugin. FingerprintingPluginManager embeds PluginManager. jobMutator and jobValidator each embed admissionController. This avoids method duplication while communicating “this is an extension of”.
  • Implicit satisfaction: Almost always consumer-defined. scheduler.State and scheduler.Planner are defined in scheduler/structs, not in nomad/ (where the concrete implementations live). DriverPlugin is defined in plugins/drivers, not in the driver implementations. This is textbook Go interface placement — the dependency always points toward the consumer package.
  • stdlib interfaces used: io.Reader/io.Writer appear in ExecOptions (driver streaming). fmt.Stringer is satisfied by various domain types. The iterator pattern is inspired by (but not directly using) database/sql row iteration.

Key abstractions#

  1. scheduler.State + scheduler.Planner — The pair of interfaces that structurally decouples the scheduling algorithms from the server. Because scheduler/ imports neither of their concrete implementations, schedulers are fully testable with in-memory fakes. This is the most consequential interface boundary in the codebase.

  2. DriverPlugin — The extension seam for task execution. Every task in every Nomad cluster ultimately flows through a DriverPlugin.StartTask() call. Its dual implementation as both a direct Go interface and a gRPC proxy (via hashicorp/go-plugin) allows the same code path to work whether the driver is in-process (tests) or out-of-process (production).

  3. FeasibleIterator / RankIterator — The composable pipeline for scheduling decisions. The iterator chain pattern allows complex scheduling logic (constraints, bin-packing, preemption, topology) to be assembled from reusable composable pieces, each independently testable and independently replaceable.

  4. StateDB — The client’s local persistence abstraction. Enables a BoltDB swap (there is an in-memory variant) and is the boundary that makes client-side state management independently testable.

  5. admissionController / jobMutator / jobValidator — The admission control chain defines how Nomad evolves the job spec processing pipeline without cluttering a single monolithic RegisterJob function. New implicit behaviors (connect sidecar injection, namespace attribute validation) are added as new hook implementations rather than edits to existing code.

Interface-driven extensibility#

Nomad uses interfaces at three distinct extensibility tiers:

Tier 1 — Internal architectural seams (scheduler.State, scheduler.Planner, StateDB): Not intended for external use, but critical for testability and clean layering. These interfaces are the reason the scheduler, server, and client can be unit-tested without standing up a full cluster.

Tier 2 — Plugin SDK (DriverPlugin, BasePlugin, device plugin interfaces in plugins/device): Intended for third-party extension. The plugins/ directory is the public SDK surface. A third-party driver author implements DriverPlugin, compiles it as a binary, and registers it in the Nomad agent config. The gRPC transport is generated from proto/driver.proto; the Go interface is the human-facing layer on top.

Tier 3 — Hook chains (admissionController, jobMutator, jobValidator): Used for internal feature composition (CE vs Enterprise behaviors). Enterprise builds add additional mutators and validators to the hook slices. Third-party code cannot add hooks without forking.

Backward Nomad — Patterns Nomad — API Surface Forward