etcd — Structure#

Layout pattern#

Multi-module Monorepo (Go Workspace)

etcd does not follow the conventional single-module Standard Go Layout (cmd/internal/pkg). Instead it uses a Go workspace (go.work) to organize 13 independent modules under one repository. This is architecturally significant: consumers of the client library (go.etcd.io/etcd/client/v3) do not transitively import any server-side code. The internal/ convention is nearly absent at the top level — boundary enforcement comes from module-level separation, not directory visibility rules.

Directory map#

repositories/etcd/
├── api/                    # Module: go.etcd.io/etcd/api/v3
│   ├── authpb/             # Protobuf: auth types (Role, User, Permission)
│   ├── etcdserverpb/       # Protobuf: core KV, Watch, Lease, Cluster gRPC service defs
│   │   └── gw/             # grpc-gateway REST JSON shim (generated)
│   ├── membershippb/       # Protobuf: cluster membership types
│   ├── mvccpb/             # Protobuf: MVCC event types (KeyValue, Event)
│   ├── v3rpc/              # RPC error codes
│   ├── version/            # Version constants (semver, GitSHA)
│   └── versionpb/          # Protobuf: cluster version negotiation
│
├── cache/                  # Module: experimental client-side watch cache
│
├── client/
│   ├── pkg/                # Module: go.etcd.io/etcd/client/pkg/v3
│   │   ├── fileutil/       # Atomic file ops, directory helpers
│   │   ├── logutil/        # Zap logger utilities
│   │   ├── pathutil/       # URL path helpers
│   │   ├── srv/            # DNS SRV record discovery
│   │   ├── systemd/        # systemd socket activation
│   │   ├── testutil/       # Shared test helpers
│   │   ├── tlsutil/        # TLS cert loading, cipher selection
│   │   ├── transport/      # HTTP transport with TLS
│   │   ├── types/          # Core types (IDs, URL sets, cluster IDs)
│   │   └── verify/         # Panic-based assertion helpers
│   └── v3/                 # Module: go.etcd.io/etcd/client/v3 (public API)
│       ├── concurrency/    # Distributed mutex, election, STM
│       ├── credentials/    # gRPC credential bundles
│       ├── experimental/   # Unstable API (e.g. watch with require-leader)
│       ├── internal/       # Endpoint parsing, retry logic internals
│       ├── kubernetes/     # Kubernetes-optimized client variant
│       ├── leasing/        # Client-side leasing cache
│       ├── mirror/         # Key-space mirroring utility
│       ├── mock/           # In-process mock client (interface-based)
│       ├── namespace/      # Key-prefix namespace isolation
│       ├── naming/         # gRPC name resolution adapter
│       ├── ordering/       # Read linearizability enforcement
│       ├── snapshot/       # Snapshot save/restore via client
│       └── yaml/           # YAML config loading
│
├── etcdctl/                # Module: go.etcd.io/etcd/etcdctl/v3
│   ├── ctlv3/              # Cobra command tree root
│   │   └── command/        # All subcommand implementations (get, put, del, watch...)
│   └── util/               # Output formatting utilities
│
├── etcdutl/                # Module: go.etcd.io/etcd/etcdutl/v3
│   ├── etcdutl/            # Cobra commands: defrag, snapshot, migrate, hashkv
│   └── snapshot/           # Snapshot manager (restore/save logic)
│
├── pkg/                    # Module: go.etcd.io/etcd/pkg/v3
│   ├── adt/                # Interval tree data structure
│   ├── cobrautl/           # Cobra exit-code utilities
│   ├── contention/         # Lock contention detection
│   ├── cpuutil/            # Byte order detection
│   ├── crc/                # CRC32 utilities
│   ├── debugutil/          # pprof HTTP handlers
│   ├── expect/             # Process-spawning test helper (exec + expect)
│   ├── featuregate/        # Runtime feature flag management
│   ├── flags/              # Custom pflag types (URL, JSON, etc.)
│   ├── grpctesting/        # gRPC test server helpers
│   ├── httputil/           # HTTP redirect, drain utilities
│   ├── idutil/             # Monotonic ID generator (generator ID + counter)
│   ├── ioutil/             # Limited reader, page writer, etc.
│   ├── netutil/            # IP resolution, interface enumeration
│   ├── notify/             # OS signal notification helpers
│   ├── osutil/             # Platform-portable file locking
│   ├── pbutil/             # Protobuf marshal/unmarshal with CRC
│   ├── proxy/              # TCP proxy and HTTP proxy (for testing)
│   ├── report/             # Benchmark reporting (percentiles, stats)
│   ├── runtime/            # Stack trace, goroutine dump utilities
│   ├── schedule/           # Job scheduler (FIFO, pausing)
│   ├── stringutil/         # String conversion helpers
│   ├── traceutil/          # Distributed tracing wrappers
│   └── wait/               # Wait groups keyed by ID, time-bounded waits
│
├── server/                 # Module: go.etcd.io/etcd/server/v3 (largest: 383 .go files)
│   ├── auth/               # RBAC: roles, users, JWT/simple token auth
│   ├── config/             # ServerConfig struct (all startup options)
│   ├── embed/              # Etcd struct — public embedding API (serve.go, etcd.go)
│   ├── etcdmain/           # Main entry point, gateway/grpc-proxy CLI root
│   ├── etcdserver/         # Core server logic
│   │   ├── api/            # Sub-APIs within the server
│   │   │   ├── etcdhttp/   # Health, metrics, version HTTP handlers
│   │   │   ├── membership/ # Cluster member management
│   │   │   ├── rafthttp/   # Raft peer HTTP transport
│   │   │   ├── snap/       # Snapshot send/receive over HTTP
│   │   │   ├── v2store/    # Legacy v2 in-memory tree store
│   │   │   ├── v3alarm/    # Alarm system (NOSPACE, CORRUPT)
│   │   │   ├── v3compactor/# Background log compaction
│   │   │   ├── v3discovery/# Bootstrap discovery (DNS, etcd-based)
│   │   │   ├── v3election/ # Leader election service
│   │   │   └── v3lock/     # Distributed lock service
│   │   │   └── v3rpc/      # gRPC service implementations (KV, Watch, Lease...)
│   │   ├── apply/          # Raft log entry application (v3 applier)
│   │   ├── cindex/         # Consistent index (applied Raft index tracking)
│   │   ├── errors/         # Server-side error sentinel values
│   │   ├── txn/            # Transaction execution engine
│   │   └── version/        # Cluster version negotiation
│   ├── features/           # Server-side feature gate definitions
│   ├── lease/              # Lease manager (TTL, renewal, attach/detach)
│   │   ├── leasehttp/      # Lease renewal HTTP proxy
│   │   └── leasepb/        # Lease protobuf types
│   ├── mock/               # Mock implementations (storage, raft) for tests
│   ├── proxy/
│   │   ├── grpcproxy/      # gRPC reverse proxy (for scaling reads)
│   │   └── tcpproxy/       # Layer-4 TCP multiplexer
│   ├── storage/            # Persistence layer
│   │   ├── backend/        # bbolt wrapper (batching, metrics, hooks)
│   │   ├── datadir/        # Data directory layout helpers
│   │   ├── mvcc/           # MVCC KV store (treeindex + bbolt)
│   │   ├── schema/         # bbolt bucket/key schema definitions
│   │   └── wal/            # Write-ahead log (encode/decode, repair)
│   └── verify/             # Data integrity verification
│
├── tests/                  # Module: go.etcd.io/etcd/tests/v3
│   ├── common/             # Shared test helpers for integration + e2e
│   ├── e2e/                # End-to-end tests (process-level)
│   ├── fixtures/           # TLS certs, config files used in tests
│   ├── framework/          # Test framework (integration + e2e harness)
│   ├── integration/        # In-process multi-member cluster tests
│   └── robustness/         # Jepsen-style linearizability + fault injection
│
├── contrib/
│   ├── lock/               # Distributed lock example
│   ├── mixin/              # Prometheus + Grafana dashboard mixin
│   ├── raftexample/        # Standalone Raft usage example (educational)
│   └── systemd/            # systemd unit file templates
│
├── tools/
│   ├── benchmark/          # Load testing tool (key/value ops)
│   ├── etcd-dump-db/       # Offline bbolt database inspection
│   ├── etcd-dump-logs/     # WAL log decoder/printer
│   ├── etcd-dump-metrics/  # Prometheus metrics scraper
│   ├── local-tester/       # Local cluster fault injection
│   ├── rw-heatmaps/        # Read/write heatmap visualization
│   └── testgrid-analysis/  # CI flakiness analysis
│
├── hack/                   # Dev scripts: TLS setup, benchmark, K8s deploy
├── scripts/                # Build, test, release, code-gen shell scripts
├── Documentation/          # Architecture docs, dev guides, postmortems
├── Makefile                # Top-level orchestration (delegates to scripts/)
├── go.work                 # Go workspace linking all 13 modules
└── Dockerfile              # Distroless image (requires pre-built binaries)

Entry points#

Binary	Main file	Role
`etcd`	`server/main.go` → `server/etcdmain/main.go`	The etcd server daemon; also serves `gateway` and `grpc-proxy` subcommands
`etcdctl`	`etcdctl/main.go` → `etcdctl/ctlv3/ctlv3.go`	Interactive CLI for all etcd operations (get, put, del, watch, lease, auth, snapshot, member, alarm, …)
`etcdutl`	`etcdutl/main.go`	Offline utility: operates directly on data files (snapshot restore, defragmentation, WAL/db inspection, migration)
`benchmark`	`tools/benchmark/main.go`	Load-testing tool
`etcd-dump-db`	`tools/etcd-dump-db/main.go`	Offline bbolt database introspection
`etcd-dump-logs`	`tools/etcd-dump-logs/main.go`	WAL file decoder
`raftexample`	`contrib/raftexample/main.go`	Educational: minimal Raft KV store using `go.etcd.io/raft/v3` directly

Package organization#

Internal packages (server-only, not meant as public API)#

server/etcdmain — startup, CLI wiring, graceful shutdown
server/etcdserver — core EtcdServer struct, Raft lifecycle, request processing
server/etcdserver/apply — applies committed Raft entries to state machine
server/etcdserver/txn — executes KV transactions
server/etcdserver/api/v3rpc — gRPC service handlers (KV, Watch, Lease, Cluster, Maintenance, Auth)
server/etcdserver/api/rafthttp — peer-to-peer Raft HTTP transport
server/etcdserver/api/membership — cluster topology (members, attributes, RaftCluster)
server/storage/mvcc — MVCC store: in-memory B-tree index + bbolt backend
server/storage/backend — batching bbolt wrapper with read views
server/storage/wal — write-ahead log encode/decode/repair
server/lease — TTL lease manager (core leasing contract)
server/auth — RBAC: role/user store, token issuers (JWT + simple)
server/proxy/grpcproxy — gRPC proxy for horizontal read scaling
server/proxy/tcpproxy — cmux-based TCP multiplexer (gRPC vs HTTP on same port)

Public packages (pkg/)#

The pkg/ module is a shared utility library consumed by both server and client modules:

pkg/featuregate — runtime feature flags (safe/unsafe, opt-in/out)
pkg/wait — ID-keyed wait groups, time-bounded waits
pkg/idutil — monotonic ID generator using generator-ID + counter
pkg/traceutil — lightweight distributed tracing (operation steps)
pkg/schedule — FIFO job scheduler with pause/resume
pkg/adt — interval tree (used for watch range overlap detection)
pkg/flags — pflag extensions for URLs, JSON, string lists
pkg/expect — subprocess “expect” for e2e testing
pkg/proxy — TCP/HTTP proxies for fault injection in tests

Public client library (client/v3)#

client/v3 is the canonical public Go API for etcd. Notable sub-packages:

client/v3/concurrency — distributed primitives: Mutex, Election, STM (software transactional memory)
client/v3/leasing — client-side read leasing cache
client/v3/namespace — transparent key prefix namespacing
client/v3/ordering — serial read enforcement (guards against stale reads)
client/v3/mock — in-process mock client implementing the same interfaces
client/v3/kubernetes — K8s-optimized client with list paging
client/v3/mirror — continuous key-space mirroring utility

Layering#

The dependency graph enforces a clean hierarchy:

tools / contrib / tests
        ↓
    etcdctl / etcdutl
        ↓
    client/v3  ←──────────────────── cache (experimental)
        ↓
    client/pkg    pkg
        ↓            ↓
       api        server  ←── (server imports api, client/pkg, pkg)
        ↓
  (generated protobuf — no Go deps above stdlib + grpc)

The server module depends on api, client/pkg, and pkg, but not on client/v3 directly (it uses a thin internal adapter server/etcdserver/api/v3client for intra-process calls). This prevents circular dependencies while allowing the server to make client-protocol calls to itself during leader elections.

Build system#

Build tool: GNU Make + Bash scripts (scripts/build.sh, scripts/build_lib.sh)
Key targets:
- make build — builds bin/etcd, bin/etcdctl, bin/etcdutl
- make tools — builds all tools binaries
- make test — runs unit + integration + release + e2e suites
- make test-robustness — Jepsen-style correctness suite
- make build-all — cross-compiles for all platforms (linux/darwin/windows × amd64/arm64/etc.)
CGO: disabled by default (pure Go, static binaries)
Version injection: -ldflags=-X=go.etcd.io/etcd/api/v3/version.GitSHA=<sha>
Docker: Yes, minimal distroless image (gcr.io/distroless/static-debian12). The Dockerfile is a simple ADD of pre-built binaries — no multi-stage build, binaries are assumed to be built outside Docker via build-docker.sh
Release: scripts/release.sh with platform matrix; no goreleaser
Code generation: scripts/genproto.sh regenerates all .pb.go files from .proto sources

Notable structural decisions#

Go Workspace as modularity boundary: The deliberate split into 13 modules enforces real separation of concerns via the Go module system rather than naming conventions. A Kubernetes operator depending on client/v3 cannot accidentally import the server’s bbolt or WAL code — the module graph prevents it. This is a sophisticated answer to the monorepo-vs-multi-repo debate.
API module as the single source of truth: All protobuf definitions live in api/, which depends only on grpc and protobuf. Both the server (implements the service) and the client (calls the service) import this module. The generated gRPC stubs are versioned along with the rest of the codebase rather than living in a separate schema repository.
embed package as the integration surface: The server/embed package (embed.Etcd, embed.Config) is the designed entry point for embedding etcd in other Go programs (Kubernetes in-process testing). This pattern avoids exposing internal server types while providing a stable integration API.
Three-binary design: The split of etcd (server daemon), etcdctl (live operations CLI), and etcdutl (offline file operations) is a deliberate safety boundary. etcdutl can operate on data files even when the server is down, and does not import network or server-startup code. This Unix-philosophy separation keeps each binary focused and reduces attack surface.
tests/ as its own module: Moving tests to a separate module (go.etcd.io/etcd/tests/v3) means test dependencies (e.g., antithesis, testcontainers-equivalent setup) don’t leak into production module graphs. The tests/robustness/ sub-package implements Jepsen-style history validation, reflecting etcd’s philosophy that correctness is non-negotiable and must be mechanically verified.
contrib/raftexample as living documentation: A complete, working key-value store built on go.etcd.io/raft/v3 lives in contrib/. This serves as both a teaching tool and a validation that the Raft module’s public API is usable independently, reinforcing the architectural separation between consensus protocol and application.