sqlc — Structure#

Layout pattern#

Standard Go Layout (cmd/internal/pkg) with Compiler-style Internal Layering

sqlc follows the idiomatic Go project layout: thin cmd/ binaries, business logic under internal/, and a minimal public pkg/ surface. The twist is that internal/ is organized as a multi-stage compiler pipeline rather than a typical service: parser engines, an AST, a catalog, a compiler, and code generators are explicit named packages that mirror the stages of a language tool. There is also a protos/ directory containing the protobuf-defined IR that formally separates the compiler core from code generation plugins.

Directory map#

sqlc/
├── cmd/                        # Binary entry points (thin wrappers)
│   ├── sqlc/                   # Main user-facing CLI binary
│   ├── sqlc-gen-json/          # JSON code-generator plugin (process-based)
│   └── sqlc-test-setup/        # CI/dev helper: installs and starts PostgreSQL & MySQL
│
├── internal/                   # All private implementation
│   ├── cmd/                    # Command implementations (generate, vet, push, diff, …)
│   ├── compiler/               # Core compilation logic: query → typed IR
│   ├── config/                 # sqlc.yaml / sqlc.json config parsing
│   │   └── convert/            # Config version migration
│   ├── codegen/                # Code generators
│   │   ├── golang/             # Built-in Go code generator
│   │   ├── json/               # Built-in JSON code generator
│   │   └── sdk/                # Shared plugin SDK utilities
│   ├── engine/                 # SQL dialect implementations
│   │   ├── postgresql/         # PostgreSQL (pg_query_go + pure-Go WASM path)
│   │   │   ├── analyzer/       # Type-checking against live PG catalog
│   │   │   ├── contrib/        # Builtin function signatures
│   │   │   └── parser/         # AST conversion from pg_query_go
│   │   ├── dolphin/            # MySQL (TiDB parser)
│   │   ├── sqlite/             # SQLite
│   │   │   ├── analyzer/
│   │   │   └── parser/
│   │   └── clickhouse/         # ClickHouse (experimental)
│   ├── sql/                    # Shared SQL AST and utilities
│   │   ├── ast/                # Language-agnostic SQL AST node types
│   │   ├── astutils/           # AST traversal helpers
│   │   ├── catalog/            # Schema catalog (tables, types, functions)
│   │   ├── format/             # SQL formatting
│   │   ├── lang/               # Query language metadata
│   │   ├── named/              # Named parameter handling
│   │   ├── rewrite/            # Query rewriting (e.g., named → positional)
│   │   ├── sqlerr/             # SQL error types
│   │   ├── sqlfile/            # SQL file reading/parsing
│   │   ├── sqlpath/            # Path resolution for SQL files
│   │   ├── validate/           # Query validation
│   │   └── info/               # Query metadata
│   ├── ext/                    # Plugin execution backends
│   │   ├── wasm/               # Wazero-based WASM plugin runner
│   │   └── process/            # Subprocess-based plugin runner
│   ├── analyzer/               # Remote/managed analyzer integration
│   ├── analysis/               # Analysis protobuf types (generated)
│   ├── bundler/                # Asset bundling for WASM plugins
│   ├── cache/                  # Plugin/result caching
│   ├── dbmanager/              # Database connection management for `vet`
│   ├── endtoend/               # End-to-end test harness + testdata/
│   ├── inflection/             # Word inflection (singularize/pluralize)
│   ├── info/                   # Version/build info
│   ├── metadata/               # SQL query annotation parsing (-- name: Foo :one)
│   ├── migrations/             # Schema migration file reading
│   ├── multierr/               # Multi-error aggregation
│   ├── opts/                   # Shared option types
│   ├── pattern/                # Glob/pattern matching for file selection
│   ├── pgx/                    # pgx connection pool helpers
│   │   └── poolcache/
│   ├── plugin/                 # Protobuf-generated plugin types (GenerateRequest/Response)
│   ├── quickdb/                # Managed cloud DB provisioning (sqlc.cloud)
│   │   └── v1/
│   ├── remote/                 # gRPC client for sqlc.cloud remote services
│   ├── rpc/                    # Internal RPC types
│   ├── shfmt/                  # Shell/SQL formatting helpers
│   ├── source/                 # Source location tracking for error messages
│   ├── sqltest/                # Test database helpers
│   │   ├── docker/             # Docker-based DB containers for tests
│   │   ├── local/              # Local DB detection
│   │   └── native/             # Native DB setup
│   ├── tracer/                 # OpenTelemetry tracing
│   ├── vet/                    # `sqlc vet` — CEL-based static analysis rules
│   ├── debug/                  # Debug output helpers
│   ├── constants/              # Shared constants
│   └── x/
│       └── expander/           # Experimental: query expansion
│
├── pkg/
│   └── cli/                    # Public CLI entrypoint (exported Do() function)
│
├── protos/                     # Protobuf source definitions
│   ├── plugin/                 # Plugin request/response IR (GenerateRequest, etc.)
│   ├── analysis/               # Analysis result types
│   └── vet/                    # Vet rule types
│
├── examples/                   # End-to-end example projects (authors, booktest, jets, …)
│   ├── authors/{mysql,postgresql,sqlite}
│   ├── batch/postgresql
│   ├── booktest/{mysql,postgresql,sqlite}
│   ├── jets/postgresql
│   └── ondeck/{mysql,postgresql,sqlite}
│
├── docs/                       # Sphinx-based user documentation
│   ├── guides/, howto/, overview/, reference/, tutorials/
│
├── scripts/                    # Build, release, and test helper scripts
│   ├── build/, bump-version/, mirror-go-plugin/,
│   ├── cleanup-test-dbs/, test-json-process-plugin/
│
├── Makefile                    # Primary build orchestration
├── Dockerfile                  # Multi-stage: golang:1.26 builder → distroless/base
└── docker-compose.yml          # PostgreSQL 16 + MySQL 9 for local test databases

Entry points#

Package organization#

Internal packages#

The internal/ tree is organized along compiler pipeline stages, each a discrete named package:

• internal/cmd — Cobra command implementations: generate.go, vet.go, push.go, diff.go, createdb.go, verify.go, parse.go
• internal/config — YAML/JSON config parsing; convert/ handles version migration
• internal/metadata — Parses query annotation comments (-- name: Foo :one) into QueryAnnotation structs
• internal/sql/ast — Language-agnostic SQL AST node types used across all engines
• internal/sql/catalog — In-memory schema catalog (tables, columns, types, functions) built from DDL
• internal/engine/{postgresql,dolphin,sqlite,clickhouse} — Dialect-specific parsers and analyzers; each converts native parse trees to the shared sql/ast
• internal/compiler — Core orchestration: drives parsing → catalog building → type resolution → IR emission
• internal/codegen/golang — Built-in Go code generator (templates, type mapping)
• internal/codegen/json — Built-in JSON plugin implementation
• internal/codegen/sdk — Helpers shared by plugin authors
• internal/ext/wasm — Wazero runtime for WASM code-gen plugins
• internal/ext/process — Subprocess execution of process-based plugins
• internal/plugin — Generated protobuf types (GenerateRequest, GenerateResponse, File)
• internal/vet — sqlc vet implementation: loads CEL rules, evaluates against query metadata
• internal/analyzer — Remote/managed type-analysis integration (sqlc.cloud)
• internal/remote — gRPC client for sqlc.cloud services
• internal/quickdb — Cloud-provisioned ephemeral databases for vet/testing
• internal/endtoend — End-to-end test runner using testdata/ golden files
• internal/sqltest — Test database provisioning: Docker, native, local detection paths
• internal/tracer — OpenTelemetry tracing integration
• internal/inflection — English word inflection for generating idiomatic Go names
• internal/migrations — Reads schema migration files in various formats
• internal/cache — Filesystem caching for plugin binaries and analysis results

Public packages (pkg/)#

• pkg/cli — Exports a single Do(args, stdin, stdout, stderr) function. This is the only public API; the cmd/sqlc/main.go delegates entirely to it. Keeping this in pkg/ rather than internal/ allows embedding sqlc as a library.

Layering#

The package dependency graph follows a clear compiler layering:

cmd/sqlc
  └── pkg/cli
        └── internal/cmd
              ├── internal/config
              ├── internal/compiler
              │     ├── internal/sql/ast, catalog, metadata
              │     └── internal/engine/{postgresql,dolphin,sqlite,clickhouse}
              ├── internal/codegen/{golang,json}   (built-in generators)
              ├── internal/ext/{wasm,process}       (plugin dispatch)
              │     └── internal/plugin              (protobuf IR)
              └── internal/vet
                    └── internal/analyzer / remote

Lower layers (ast, catalog, engine) have no dependency on upper layers (codegen, vet, ext). The protobuf IR in internal/plugin is the formal boundary between the compiler core and code generators, whether built-in or external.

Build system#

• Build tool: make + standard go build
• Key targets:
  • make build — go build ./... (compiles everything)
  • make install — installs all binaries to $GOPATH/bin
  • make test — unit tests only (no database required)
  • make test-examples — go test --tags=examples ./... (requires live databases)
  • make test-ci — test-examples + build-endtoend + go vet
  • make proto — regenerates protobuf code via buf generate
  • make sqlc-dev — builds sqlc to ~/bin/sqlc-dev for local iteration
  • make start — docker compose up -d (starts PG + MySQL for tests)
• Protobuf: buf (Buf toolchain) generates Go code from protos/; protoc used for the remote gRPC proto
• Release: scripts/release.go (custom Go script, invoked by Dockerfile)
• Docker: Yes, multi-stage. Stage 1: golang:1.26.1 builder runs scripts/release.go -docker. Stage 2: gcr.io/distroless/base-debian12 — minimal final image with no shell.
• CI: GitHub Actions (.github/workflows/ci.yml); uses sqlc-test-setup binary instead of Docker to provision databases directly on the runner; runs gotestsum for JUnit output.

Notable structural decisions#

1. pkg/cli as the only public package — All logic lives in internal/; pkg/cli exports only Do(). This makes the project embeddable as a library while strongly protecting internal APIs from external consumers.

2. Protobuf IR as the plugin contract — Rather than a Go interface, the boundary between the compiler and code generators is a .proto-defined GenerateRequest/Response. This enables the plugin ecosystem (WASM or process-based) to be language-agnostic: any language that can speak protobuf can be a sqlc code generator.

3. Engine isolation under internal/engine/ — Each SQL dialect (PostgreSQL, MySQL/Dolphin, SQLite, ClickHouse) has its own sub-package with its own parser and analyzer. They all convert to the shared internal/sql/ast types, cleanly separating dialect knowledge from the generic compiler.

4. internal/endtoend/testdata/ as a golden-file corpus — The end-to-end test suite is driven by a large directory of testdata/ examples rather than hand-written test functions. This is both the regression test suite and living documentation of every supported SQL dialect and config variation.

5. Three plugin execution modes — sqlc supports code-gen plugins as WASM modules (Wazero), as subprocesses (stdin/stdout protobuf), or as built-in Go packages (internal/codegen/golang). The internal/ext/ tree cleanly separates these three dispatch strategies without changing the rest of the compiler.

6. sqlc-test-setup as a first-class binary — Instead of requiring Docker in CI, the project ships its own database provisioner as a cmd/ binary that downloads PostgreSQL binaries and installs MySQL via apt. This reflects a philosophy of making the project self-contained for contributors.