CockroachDB — Structure#

Layout pattern#

Custom Monorepo (Domain-Oriented pkg/ with CCL Separation)

CockroachDB does not follow standard Go layout conventions. All Go source code lives under a single pkg/ directory organized by functional domain, not by internal/external visibility. There is no top-level internal/ directory (the pkg/internal/ package exists but is minor). The key structural innovation is the CCL separation: commercial/enterprise features are co-located in pkg/ccl/ and injected into the main binary via blank imports (_), while an OSS binary (cockroach-short) omits the CCL import. A massive build system (Bazel + dev wrapper) manages the enormous dependency and code-generation graph.

Directory map#

cockroach/
├── build/                  # Build infrastructure
│   ├── bazelbuilder/       #   Docker image for Bazel CI workers
│   ├── bazelutil/          #   Bazel utility scripts
│   ├── bootstrap/          #   One-time dev environment setup
│   ├── deploy/             #   Production Docker image (UBI Minimal base)
│   ├── deploy-redhat/      #   Red Hat certified image
│   ├── ghactions/          #   GitHub Actions workflows
│   ├── release/            #   Release pipeline scripts
│   └── teamcity/           #   TeamCity CI pipeline scripts
│
├── c-deps/                 # Bundled C/C++ native dependencies (CGo)
│   ├── geos/               #   GEOS for geospatial SQL functions
│   ├── jemalloc/           #   jemalloc memory allocator
│   ├── krb5/               #   Kerberos for enterprise auth
│   └── proj/               #   PROJ coordinate transformation library
│
├── cloud/
│   └── kubernetes/         # Kubernetes Helm charts and operator manifests
│
├── docs/
│   ├── RFCS/               # Design proposals (RFCs) — key architectural decisions
│   ├── tech-notes/         # Deep-dive technical documentation
│   └── tla-plus/           # TLA+ formal specs for Raft and transactions
│
├── licenses/               # Third-party license files
│
├── monitoring/
│   ├── grafana-dashboards/ # Grafana JSON dashboards for metrics
│   └── rules/              # Prometheus alerting rules
│
├── pkg/                    # ALL Go source code (see Package Organization below)
│
├── scripts/                # Misc shell scripts (CI, release, formatting, linting)
│
├── tools/                  # Build toolchain helpers
│
├── WORKSPACE               # Bazel workspace root
├── DEPS.bzl                # External Go dependency declarations for Bazel
├── GNUmakefile             # Thin wrapper — delegates to `./dev`
├── dev                     # Primary developer CLI (Bazel orchestrator, self-building)
└── go.mod                  # Go module file (required even with Bazel)

Entry points#

CockroachDB has an unusually large number of binaries in pkg/cmd/. The most significant:

Primary Production Binaries#

Developer / Test Tooling#

Research / Specialized Tools#

Package organization#

Core Database Stack (bottom to top)#

pkg/kv/ Sub-Package Structure#

pkg/kv/
├── kvclient/               # Client-side KV API used by SQL layer
│   ├── kvcoord/            #   DistSender: routes KV requests to correct ranges
│   ├── kvstreamer/         #   Streaming KV reads for vectorized execution
│   ├── kvtenant/           #   Multi-tenant KV client
│   ├── rangecache/         #   Cache of range descriptors and leases
│   └── rangefeed/          #   Change feed subscriptions at KV level
├── kvserver/               # KV store: Raft replicas, range management
│   ├── allocator/          #   Range replica placement decisions
│   ├── apply/              #   Raft log application pipeline
│   ├── batcheval/          #   Evaluation of KV batch commands
│   ├── concurrency/        #   Lock table, transaction wait queue
│   ├── kvflowcontrol/      #   Replication admission control
│   ├── liveness/           #   Node liveness heartbeats
│   └── split/              #   Range split decision making
├── kvpb/                   # KV protocol buffer definitions
└── bulk/                   # Bulk ingestion for import/backup

pkg/sql/ Sub-Package Structure (sampled key areas)#

pkg/sql/
├── parser/                 # Hand-written SQL parser (not yacc/antlr)
├── sem/tree/               # AST node definitions for the SQL language
├── opt/                    # Cascades-style cost-based optimizer
│   ├── optbuilder/         #   Builds optimizer memo from AST
│   ├── xform/              #   Transformation rules (exploration)
│   └── exec/               #   Execution plan building
├── execinfra/              # DistSQL execution infrastructure interfaces
├── execinfrapb/            # DistSQL execution plan proto definitions
├── colexec/                # Vectorized execution operators (columnar)
├── colflow/                # Columnar DistSQL flow orchestration
├── catalog/                # Descriptor management (tables, indexes, schemas)
│   ├── descs/              #   Descriptor collection / leasing
│   └── lease/              #   Schema object lease management
├── schemachanger/          # Declarative schema change engine
│   └── scplan/             #   Schema change planning (dependency graph)
├── pgwire/                 # PostgreSQL wire protocol handler
├── distsql/                # Distributed SQL execution coordinator
└── sqlstats/               # Statement/transaction statistics collection

Supporting Packages#

pkg/internal/ (Go-conventional internal)#

pkg/internal/
├── client/         # Internal test client helpers
├── codeowners/     # CODEOWNERS file parser
├── metricscan/     # Metrics registry scanner
├── rsg/            # Random syntax generator (SQL fuzzing)
├── sqlsmith/       # SQLsmith random query generator
└── team/           # Team ownership utilities

Layering#

The dependency flow is strictly bottom-up:

pkg/cli (top: user interface)
  └─ pkg/server (node lifecycle, gRPC endpoints)
       └─ pkg/sql (SQL engine, query processing)
            └─ pkg/kv/kvclient (distributed KV client)
                 └─ pkg/kv/kvserver (Raft, range management)
                      └─ pkg/storage (Pebble MVCC engine)
                           └─ pkg/raft (consensus algorithm)
                                └─ pkg/roachpb (protocol types)
                                     └─ pkg/util (shared utilities)

Cross-cutting concerns (tracing, logging, settings, security) live in pkg/util/, pkg/settings/, and pkg/security/ and are imported by all layers.

Build system#

• Primary tool: Bazel (with io_bazel_rules_go for Go support)
• Developer interface: ./dev — a self-building Go binary (pkg/cmd/dev/) that orchestrates Bazel commands. Developers never invoke bazel directly.
• GNUmakefile: Thin wrapper — all targets delegate to ./dev build <target>.
• Key build targets:
  • ./dev build cockroach — full binary with CCL + web UI
  • ./dev build short — binary without web UI (faster builds)
  • ./dev build cockroach-sql — SQL shell only
  • ./dev test pkg/kv/... — run tests for a package subtree
  • ./dev generate — run code generation (protos, Bazel BUILD files, etc.)
  • ./dev bench pkg/sql/... — run benchmarks
• Go module: go.mod is present (required for tooling compatibility) but Bazel’s DEPS.bzl is authoritative for hermetic builds.
• No vendor/ directory: Dependencies are declared in DEPS.bzl and fetched by Bazel. The standard go mod vendor workflow is not used.
• Docker: Deployment image (build/deploy/Dockerfile) uses Red Hat UBI Minimal as base — a single-stage Dockerfile that copies a pre-built binary into a minimal runtime image. Not a multi-stage build; compilation happens outside Docker via Bazel.
• CGo: Enabled for GEOS, jemalloc, KRB5, PROJ. The c-deps/ sources are compiled via Bazel C++ rules and linked in.
• Code generation: Protos (.proto → *pb.go), Bazel BUILD files (gazelle), vectorized engine operators (execgen), optimizer rules (optgen), and SQL built-in definitions are all generated.

Notable structural decisions#

1. pkg/ monolith with no enforced visibility at directory level. Unlike projects that separate internal/ (unexported) from pkg/ (exported for third-party use), CockroachDB’s pkg/ is entirely internal to the module. Callers outside the repository cannot import it. This is a pragmatic choice for a large application codebase that is not meant to be a library.

2. CCL injection via blank imports. The OSS and commercial binaries share a codebase. Enterprise features in pkg/ccl/ register themselves via init() functions. The commercial binary imports pkg/ccl with a blank import (_ "github.com/cockroachdb/cockroach/pkg/ccl"), activating all CCL features at startup. This avoids build tags or conditional compilation and keeps the layering clean.

3. Custom Raft in pkg/raft/. Rather than adopting etcd/raft (the near-universal choice in Go distributed systems), CockroachDB maintains its own Raft implementation forked from etcd’s, with deep integration to HLC clocks, MultiRaft optimizations (a single Raft state machine per store serving many ranges), and their own storage liveness protocol.

4. SQL engine dwarfs the KV engine in file count (2361 vs 605). The pkg/sql/ subtree rivals standalone SQL database engines in scope — it contains a custom parser, a Cascades-style cost-based optimizer, a vectorized execution engine, a declarative schema change planner, and a full PostgreSQL wire protocol implementation. This reflects the philosophy that SQL compatibility is a first-class concern, not an afterthought.

5. Bazel build graph manages a complex multi-language codebase. The choice of Bazel (over standard go build) enables: (a) hermetic, reproducible builds across Go, C/C++, and proto code; (b) remote caching for CI; (c) fine-grained dependency tracking for incremental builds. The cost is significant build system complexity — every package has a BUILD.bazel file maintained via Gazelle.

6. Separate tools binary ecosystem. Beyond the production cockroach binary, the project ships ~20 developer/operator tools as separate binaries (roachprod, roachtest, workload, dev, smith, reduce, etc.). These are first-class citizens of the repo, not afterthoughts. This reflects the maturity of the project’s engineering infrastructure.

7. Proto-first protocol definition throughout. Every major subsystem defines its wire format in .proto files with generated *pb sub-packages (kvpb, roachpb, serverpb, kvserverpb, execinfrapb, etc.). This creates a clear boundary between serialized data (proto) and behavior (Go types wrapping proto types).