Consul — Structure#

Layout pattern#

Custom Monorepo (not Standard Go Layout)

Consul deviates from the canonical cmd/-based Go layout. There is a single main.go at the root that produces one consul binary. The project is structured as a multi-module monorepo: five sub-modules (api/, sdk/, proto-public/, envoyextensions/, troubleshoot/) have their own go.mod files and are referenced from the root via replace directives. The dominant package organizing principle is by feature domain (agent/, acl/, connect/, command/) rather than by layer.

Directory map#

consul/
├── main.go                    # Single entry point for the consul binary
├── acl/                       # ACL policy engine and token evaluation
│   └── resolver/              # ACL resolver implementations
├── agent/                     # Core agent logic (~40 sub-packages; the heart of the project)
│   ├── ae/                    # Anti-entropy sync (local vs. server state)
│   ├── auto-config/           # Automatic configuration via JWT/TLS bootstrapping
│   ├── cache/                 # Client-side caching layer with blocking queries
│   ├── checks/                # Health check runners (HTTP, TCP, script, gRPC, etc.)
│   ├── config/                # Configuration parsing and validation
│   ├── configentry/           # Config entry management (service defaults, etc.)
│   ├── connect/               # Agent-side Connect/service-mesh logic
│   ├── consul/                # Server-mode agent: Raft FSM, RPC handlers, state store
│   ├── dns/                   # DNS server implementation
│   ├── grpc-external/         # gRPC services exposed externally (v1 and v2)
│   ├── grpc-internal/         # Internal gRPC (server-to-server)
│   ├── grpc-middleware/       # gRPC interceptors (rate limiting, ACL, logging)
│   ├── leafcert/              # Leaf certificate manager for Connect TLS
│   ├── local/                 # Local state manager (services/checks registered on this agent)
│   ├── proxycfg/              # Proxy configuration watchers (feeds xDS to Envoy)
│   ├── proxycfg-glue/         # Glue layer connecting proxycfg to storage backends
│   ├── proxycfg-sources/      # Data sources for proxy config (catalog, config entries)
│   ├── router/                # Server routing and area management
│   ├── rpc/                   # RPC client pool and load balancing
│   ├── rpcclient/             # High-level RPC client wrappers
│   ├── structs/               # Shared data structures (the project's "types" package)
│   ├── token/                 # Token store and management
│   ├── uiserver/              # Embedded web UI HTTP server
│   └── xds/                   # Envoy xDS (v2/v3) delta and full-state servers
├── api/                       # [Sub-module] Public Go client library for Consul HTTP API
│   └── watch/                 # Watch plan helpers
├── bench/                     # Benchmarks (config parsing, etc.)
├── build-support/             # Docker files, shell scripts, build tooling
├── command/                   # CLI command implementations (~35 commands)
│   ├── acl/                   # ACL token/policy/role CRUD commands
│   ├── agent/                 # `consul agent` command
│   ├── catalog/               # Catalog query commands
│   ├── connect/               # Connect intention commands
│   ├── kv/                    # KV store commands
│   ├── operator/              # Raft/autopilot operator commands
│   ├── peering/               # Cluster peering commands
│   ├── snapshot/              # Snapshot save/restore commands
│   ├── tls/                   # TLS cert generation commands
│   ├── flags/                 # Shared flag definitions
│   ├── registry.go            # Central command registry
│   └── ...                    # ~25 more command sub-packages
├── connect/                   # Connect service mesh: cert generation, proxy protocol
│   ├── certgen/               # Certificate generation utilities
│   └── proxy/                 # Proxy configuration types
├── docs/                      # Architecture and design documentation (markdown)
│   └── v2-architecture/       # Docs for the new resource-based v2 architecture
├── envoyextensions/           # [Sub-module] Envoy extension framework
│   ├── extensioncommon/       # Base types for Envoy extensions
│   └── xdscommon/             # Shared xDS utilities for extensions
├── grafana/                   # Grafana dashboard JSON definitions
├── grpcmocks/                 # Generated gRPC mocks for proto-public
├── internal/                  # Internal-only packages (not importable externally)
│   ├── controller/            # Resource controller framework (v2 architecture)
│   │   ├── cache/             # Controller cache with indexing
│   │   └── dependency/        # Dependency tracking for controllers
│   ├── dnsutil/               # DNS utilities
│   ├── gossip/                # Gossip/Serf and RTT library wrappers
│   ├── go-sso/                # OIDC/SSO authentication library (vendored-in)
│   ├── multicluster/          # Multi-cluster / federation resource types and controllers
│   ├── protohcl/              # Protobuf <-> HCL conversion
│   ├── protoutil/             # Protobuf utility helpers
│   ├── radix/                 # Radix tree implementation
│   ├── resource/              # Resource system core (v2 architecture CRUD framework)
│   ├── resourcehcl/           # HCL parsing for resource API
│   ├── storage/               # Storage backends: in-memory and Raft-backed
│   │   ├── inmem/             # In-memory storage (tests, dev mode)
│   │   └── raft/              # Raft FSM storage backend
│   ├── testing/               # Internal test helpers (golden files, error helpers)
│   └── tools/                 # protoc plugins (consul-rate-limit, grpc-clone, rpc-glue)
├── ipaddr/                    # IP address utilities
├── lib/                       # Small reusable utility packages
│   ├── channels/              # Channel helpers
│   ├── decode/                # MapStructure decode helpers
│   ├── file/                  # File I/O utilities
│   ├── maps/                  # Map utilities
│   ├── mutex/                 # Mutex utilities
│   ├── retry/                 # Retry/backoff logic
│   ├── routine/               # Goroutine lifecycle management
│   ├── semaphore/             # Semaphore implementation
│   ├── stringslice/           # String slice helpers
│   ├── template/              # Template helpers
│   ├── ttlcache/              # TTL-based cache
│   └── testhelpers/           # Test utilities
├── logging/                   # Logging setup (hclog-based)
│   └── monitor/               # Log streaming monitor
├── proto/                     # Generated protobuf (private wire formats)
│   └── private/               # Internal proto types
├── proto-public/              # [Sub-module] Public protobuf API definitions
│   ├── pbacl/                 # ACL proto types
│   ├── pbconnectca/           # Connect CA proto types
│   ├── pbdataplane/           # Dataplane proto types
│   ├── pbdns/                 # DNS proto types
│   ├── pbmulticluster/        # Multi-cluster proto types
│   ├── pbresource/            # Resource API proto types (v2)
│   └── pbserverdiscovery/     # Server discovery proto types
├── sdk/                       # [Sub-module] Lightweight SDK (safe for external use)
│   ├── freeport/              # Free port finder (test helper)
│   ├── iptables/              # iptables manipulation for transparent proxy
│   └── testutil/              # Test server helpers
├── sentinel/                  # Sentinel policy evaluation stubs (enterprise feature hook)
├── service_os/                # OS-specific service management (Windows service support)
├── snapshot/                  # Snapshot serialization/deserialization
├── test/                      # Test fixtures, certs, and integration test infra
│   └── integration/           # Integration test containers (consul-container framework)
├── test-integ/                # End-to-end integration tests
│   ├── connect/               # Connect/service-mesh E2E tests
│   ├── peering_commontopo/    # Peering topology E2E tests
│   └── upgrade/               # Upgrade compatibility E2E tests
├── testing/
│   └── deployer/              # [Sub-module] Topology deployment framework for E2E tests
├── testrpc/                   # Test RPC helper (starts test servers)
├── tlsutil/                   # TLS configuration and certificate utilities
├── tools/
│   └── internal-grpc-proxy/   # Internal gRPC proxy tool (main.go)
├── troubleshoot/              # [Sub-module] Troubleshooting commands and proxy validation
├── types/                     # Shared fundamental types (NodeID, ServiceID, etc.)
├── ui/                        # Ember.js web UI (non-Go)
└── version/                   # Version info (injected via ldflags)

Entry points#

BinaryFileDescription
consulmain.goThe single unified binary — delegates to command.RegisteredCommands() via mitchellh/cli. Acts as agent, server, CLI tool, and dev-mode server.
internal-grpc-proxytools/internal-grpc-proxy/main.goInternal development/debugging tool: transparent gRPC proxy for inspecting inter-server traffic.

The root main.go is intentionally thin: it sets up a mitchellh/cli.CLI instance, registers all commands from command/registry.go, and dispatches. No framework magic; the entire command surface is visible in one file.

Package organization#

  • Internal packages (internal/):

    • internal/controller — Resource controller runtime for the v2 architecture; manages reconciliation loops for resource types
    • internal/storage — Storage abstraction with in-memory and Raft-backed backends
    • internal/resource — Resource CRUD framework (v2 API system)
    • internal/multicluster — Multi-cluster federation resource types
    • internal/gossip — Wrappers over HashiCorp’s Serf/memberlist gossip libraries
    • internal/go-sso — Vendored OIDC/SSO auth library (inlined to avoid external dependency)
    • internal/tools — Custom protoc code generators (not runtime packages)

  • Public packages (pkg/-equivalent; exported but no pkg/ directory):

    • api/ — Full HTTP client library for Consul (its own Go module; widely consumed externally)
    • sdk/ — Minimal SDK with iptables, test utilities, and port helpers (own module)
    • proto-public/ — Public protobuf-generated types for gRPC APIs (own module)
    • envoyextensions/ — Envoy xDS extension framework (own module)
    • connect/ — Service mesh types and proxy protocol helpers
    • acl/ — ACL policy engine (imported by many packages)
    • tlsutil/ — TLS configuration helpers
    • lib/ — Collection of small utility packages (retry, routine, semaphore, etc.)

  • Layering: Consul uses a domain-driven, not a clean/hexagonal architecture. The layering is roughly:

    1. lib/, types/, ipaddr/ — pure utilities, no consul deps
    2. acl/, tlsutil/, logging/ — infrastructure concerns
    3. agent/structs/ — shared data model (imported by almost everything)
    4. agent/consul/ — server core (Raft FSM, state store, RPC handlers)
    5. agent/ — agent runtime (health checks, DNS, cache, proxycfg)
    6. command/ — CLI wrappers around agent functionality

    There is notable layering violation: agent/structs/ acts as a “god package” for shared types, creating a diamond dependency across the codebase.

Build system#

  • Build tool: GNU Make (Makefile at root), with go build as the underlying compiler. Uses ldflags to inject GitCommit and BuildDate via the version package.
  • Key targets:
    • make dev — build development binary for the current platform
    • make test — run unit tests via gotestsum
    • make generate — regenerate protobuf types (uses buf, custom protoc plugins)
    • make lint — run golangci-lint
    • make docker — build Docker images
  • Docker: Yes, multi-stage (Dockerfile at root with four named targets):
    • official — downloads pre-built release from releases.hashicorp.com (Alpine-based)
    • default — uses CI-built binary copied from dist/ (Alpine-based)
    • dev — for local development
    • ubi — Red Hat Universal Base Image (OpenShift-compatible)
  • Ports exposed in Docker: 8300 (server RPC), 8301/8302 (Serf LAN/WAN gossip), 8500 (HTTP API), 8600 (DNS)
  • Proto tooling: buf for proto linting/generation; custom protoc plugins in internal/tools/ for rate-limit annotations, gRPC clone generation, and RPC glue

Notable structural decisions#

  1. Root-level main.go instead of cmd/: Consul predates the cmd/-per-binary convention being common. The single root main.go reflects that there is truly one binary doing everything — and it’s never needed to change.

  2. Multi-module monorepo with replace directives: api/, sdk/, proto-public/, envoyextensions/, and troubleshoot/ are independent Go modules pinned via replace in the root go.mod. This allows consumers to import just consul/api without the full server dependency tree — a practical split for the most widely-used client library.

  3. Two parallel architectures (“v1” and “v2”): The internal/resource/, internal/controller/, and internal/storage/ packages represent an emerging resource-oriented v2 architecture (documented in docs/v2-architecture/). The existing agent/consul/ (Raft FSM + state store + RPC handlers) is the v1 architecture. Both coexist in the same binary, indicating an active long-running migration.

  4. agent/structs/ as the universal data model: Rather than thin DTOs at API boundaries, Consul centralizes nearly all wire and storage types in agent/structs/. This creates high coupling but eliminates translation layers — a pragmatic choice for a project that evolved incrementally.

  5. Inlined vendor (internal/go-sso/): The OIDC/SSO library is vendored inside internal/go-sso/ rather than as a go.sum dependency. This is unusual and suggests it diverged significantly from any upstream, or needed private modifications.

  6. Dedicated integration test infrastructure: testing/deployer/ (its own Go module) is a full topology deployment framework for spinning up multi-node Consul clusters in containers, used by test-integ/ E2E tests. This is more sophisticated than typical integration test setups and reflects the operational complexity of testing a distributed system.

  7. Custom protoc generators as first-class tools: internal/tools/protoc-gen-consul-rate-limit and protoc-gen-grpc-clone are purpose-built code generators checked into the repo, not third-party tools. This allows rate limiting and clone semantics to be expressed in .proto annotations rather than handwritten per-RPC.

Backward Consul — Overview Consul — Dependencies Forward