Nomad — Structure#

Layout pattern#

Custom / Domain-Driven Layout (not standard Go cmd/internal/pkg)

Nomad uses a domain-driven layout where top-level directories represent major functional subsystems rather than the conventional cmd/, internal/, pkg/ trio. There is no cmd/ directory — the single main.go lives at the repo root. The primary server code lives in nomad/, client code in client/, scheduler in scheduler/, and so on. This mirrors HashiCorp’s house style seen across Vault and Consul: large, named-by-purpose packages at the root level, with a shared helper/ package providing cross-cutting utilities.

Directory map#

nomad/                        # Root — single binary project
├── main.go                   # Entry point: CLI dispatcher + subcommand init-hooks
├── acl/                      # ACL policy engine (evaluation, not HTTP endpoints)
├── api/                      # Standalone Go module: public client library (go.mod)
│   ├── contexts/             # Context key constants for API requests
│   └── internal/             # Internal helpers for the API module
├── client/                   # Nomad agent client subsystem
│   ├── allocdir/             # Allocation directory management (volumes, chroots)
│   ├── allochealth/          # Alloc health tracking (checks + services)
│   ├── allocrunner/          # Core: runs a single allocation (hooks, task lifecycle)
│   ├── allocwatcher/         # Watches remote alloc state for migration
│   ├── commonplugins/        # Shared plugin initialization for client
│   ├── config/               # Client configuration struct + parsing
│   ├── consul/               # Consul service registration on client side
│   ├── devicemanager/        # Plugin manager for device drivers (GPU, etc.)
│   ├── dynamicplugins/       # Registry for dynamically-loaded plugins (CSI)
│   ├── fingerprint/          # System fingerprinting (CPU, memory, network)
│   ├── hoststats/            # Host resource stats collector
│   ├── hostvolumemanager/    # Host volume lifecycle management
│   ├── interfaces/           # Client-internal interfaces (AllocRunner, etc.)
│   ├── lib/                  # Client-internal utilities
│   ├── logmon/               # Log monitoring process (runs as child subprocess)
│   ├── pluginmanager/        # Plugin lifecycle management base
│   ├── servers/              # Client-side server list and load balancing
│   ├── serviceregistration/  # Abstraction over Consul/Nomad service registration
│   ├── state/                # Client state persistence (BoltDB)
│   ├── structs/              # Client-specific structs
│   ├── taskenv/              # Task environment variable interpolation
│   ├── testutil/             # Client test helpers
│   ├── vaultclient/          # Vault token management on client side
│   └── widmgr/               # Workload Identity manager
├── command/                  # CLI command implementations + HTTP API (agent)
│   ├── agent/                # HTTP API server, agent lifecycle, config parsing
│   └── ui/                   # Embedded web UI assets
├── drivers/                  # Built-in task drivers
│   ├── docker/               # Docker driver + docker logger subprocess
│   ├── exec/                 # Linux exec driver (namespaces/cgroups)
│   ├── java/                 # Java driver
│   ├── mock/                 # Mock driver for testing
│   ├── qemu/                 # QEMU VM driver
│   ├── rawexec/              # Raw exec driver (no isolation)
│   └── shared/               # Shared driver utilities (executor, env)
├── e2e/                      # End-to-end test suites (~40 scenarios)
├── helper/                   # Cross-cutting utility packages (~45 sub-packages)
├── internal/                 # Internal test helpers
├── jobspec2/                 # HCL2 job specification parser
├── lib/                      # Shared algorithms (auth, heaps, file utils)
├── nomad/                    # Server subsystem: Raft FSM, RPC endpoints, scheduler workers
│   ├── auth/                 # Server-side authentication/authorization
│   ├── deploymentwatcher/    # Deployment health and promotion logic
│   ├── drainer/              # Node drain orchestration
│   ├── lock/                 # Distributed locking subsystem
│   ├── mock/                 # Server mock data for tests
│   ├── peers/                # Raft peer management helpers
│   ├── reporting/            # Usage telemetry/reporting
│   ├── state/                # Raft state store (memdb)
│   ├── stream/               # Event streaming (subscribe/publish)
│   ├── structs/              # Core domain structs (Job, Alloc, Node, …)
│   └── volumewatcher/        # CSI volume claim watcher
├── plugins/                  # Plugin SDK (task drivers, device, CSI)
│   ├── base/                 # Base plugin interface + fingerprinting
│   ├── csi/                  # CSI plugin interface
│   ├── device/               # Device plugin interface
│   ├── drivers/              # Task driver plugin interface
│   └── shared/               # Shared plugin RPC utilities
├── scheduler/                # Standalone scheduler (bin-packing, spread, system)
│   ├── benchmarks/           # Scheduler performance benchmarks
│   ├── feasible/             # Constraint-based feasibility checking
│   ├── integration/          # Scheduler integration tests
│   ├── reconciler/           # Deployment reconciliation logic
│   ├── structs/              # Scheduler-specific structs
│   └── tests/                # Scheduler unit tests
├── testutil/                 # Top-level test helpers (server/client bringup)
├── tools/                    # Developer tooling (changelog, proto tools)
├── ui/                       # Ember.js web UI (separate build, embedded via go:embed)
├── version/                  # Version constants and build metadata
└── website/                  # Documentation source (not Go code)

Entry points#

Nomad ships as a single binary — there is no cmd/ directory. The single entry point is:

  • main.go — CLI dispatcher using github.com/hashicorp/cli. Dispatches to subcommands in command/. Importantly, it also imports several subprocess packages with _ (blank identifier) so their init() functions run; this allows the same binary to function as a parent process or as logmon, executor, and docker_logger subprocesses by inspecting os.Args[0] or environment variables at startup.

Key subcommands defined in command/:

  • agent — starts a Nomad server and/or client node
  • run / stop / status — job lifecycle management
  • alloc / job / node — resource inspection
  • operator — Raft management, snapshot, autopilot
  • namespace, acl, keyring, scaling, volume — administrative commands

Package organization#

Internal packages (no internal/ convention)#

Nomad places its internal subsystems directly at root level — there is no internal/ directory enforcing import restrictions (except a tiny internal/testing/ helper). Internal-by-convention packages:

  • nomad/ — Server: Raft FSM, all RPC endpoint handlers, scheduler worker goroutines, eval broker, plan queue. The nomad/structs/ sub-package is the canonical domain model for the entire codebase.
  • client/ — Client: allocation runner, plugin manager, fingerprinter, state persistence.
  • scheduler/ — Standalone scheduling algorithms; no server/client dependencies.
  • drivers/ — Built-in task drivers, each as its own package, using the plugins/drivers interface.
  • command/agent/ — HTTP API server (REST + WebSocket). Sits between external callers and the nomad/ RPC layer.
  • helper/ — ~45 small utility packages covering crypto, TLS, logging, codec, flags, broker, etc.

Public packages (importable by external tools)#

  • api/ — First-class public Go module (github.com/hashicorp/nomad/api). Client library for interacting with the Nomad HTTP API. Maintained with separate go.mod.
  • plugins/ — Plugin SDK interfaces: drivers, devices, CSI. External driver authors implement these.

Layering#

The dependency hierarchy flows roughly:

main.go
  └── command/          (CLI, HTTP API)
        └── nomad/      (Server: Raft, RPC, eval broker)
              ├── scheduler/   (scheduling algorithms — no server imports)
              ├── nomad/state/ (memdb state store)
              └── client/      (client agent — separate logical node)
                    └── drivers/ (task execution via plugins/)
plugins/              (shared interface layer — imported by both client/ and drivers/)
nomad/structs/        (domain model — imported by almost everything)
helper/               (utilities — imported by almost everything)

nomad/structs/ is the shared domain model and the most widely-imported package in the repo — a deliberate central coupling point for Job, Allocation, Node, and Evaluation types.

Build system#

  • Build tool: GNU Make (GNUmakefile) — primary build orchestration
  • Key targets:
    • make dev — build the nomad binary with development flags
    • make release — cross-compile for all platforms (linux amd64/arm64/s390x, darwin, windows, freebsd)
    • make test — run test groups (nomad, client, command, drivers, quick) via GOTEST_GROUP
    • make generate — run go generate for protobuf, stringer, and mock generation
    • make proto — regenerate protobuf bindings via buf
    • make ui — build the Ember.js UI and embed it
    • make docker — build the release Docker image
  • Build tags: ui (embeds the web UI), hashicorpmetrics, codegen_generated (CI), ent (enterprise edition)
  • Docker: Yes, multi-stage. Stage 1 (alpine) adds tzdata; stage 2 (busybox) is the final minimal runtime image. The binary is injected externally (not built inside Docker).

Notable structural decisions#

  1. Single binary, multi-role subprocess pattern. The same nomad binary acts as CLI, server, client agent, and also as child subprocess implementations (logmon, executor, docker_logger). The init() trick at the top of main.go drops into subprocess logic before the CLI framework initializes, avoiding memory overhead from unused imports. This is a sophisticated use of Go init-ordering that is uncommon at this scale.

  2. api/ as a separate Go module. The public client library is a distinct module with its own go.mod. This enforces that external tooling can depend on the API client without pulling in the entire server binary’s transitive dependencies (Raft, Consul SDK, etc.). The main module references it via a replace directive during development.

  3. nomad/structs/ as the monolithic domain model. Rather than distributing domain types across packages, all core types (Job, TaskGroup, Allocation, Node, Evaluation, Deployment, etc.) live in one large package. This creates broad coupling but also a single authoritative source of truth — a pragmatic choice for a system with many cross-cutting concerns.

  4. scheduler/ isolated from nomad/ (server). The scheduling algorithms have no import dependency on the server package. They consume and produce structs types and operate through an explicit State interface. This enables isolated unit testing of scheduling logic and, in principle, alternative scheduler implementations.

  5. Plugin system as first-class SDK. The plugins/ directory provides a versioned, interface-defined SDK for task drivers, device plugins, and CSI plugins — not just internal use but designed for external plugin authors. Combined with hashicorp/go-plugin (RPC-based subprocess plugins), this creates a clear extension boundary that third-party driver authors can target without forking Nomad.

  6. e2e/ as a comprehensive integration suite. With ~40 named test scenarios covering ACLs, Consul/Vault integration, CSI volumes, networking, scaling, disconnected clients, and more, the e2e suite is a first-class artifact — it even has its own framework/ sub-package for test orchestration and Terraform configs for provisioning test clusters.

Backward Nomad — Overview Nomad — Dependencies Forward