Decision Trees for Go Architecture: A Practitioner’s Guide from Fifty-One Projects#

Orientation#

The preceding chapters map the terrain: nine architectural archetypes, four scale tiers, and the evolutionary arc from ad-hoc to structured Go. This chapter asks a different question: given where you are right now — a project with specific requirements, constraints, and team size — what should you do next?

Decision trees are a crude tool for a complex discipline. No flowchart captures every nuance, and every real project has special cases. What makes these trees useful is that they are empirically derived: each branch corresponds to a pattern chosen by actual production systems, and the corpus provides counterexamples for every path not taken. The goal is not to eliminate judgment but to short-circuit the most common false starts — the patterns adopted too early, the migrations deferred too long, the defaults that work at one scale and break at the next.

Seven decisions dominate Go architecture. Each is examined below as a decision tree grounded in evidence from the corpus.

Decision 1: Project Layout#

The question: How should I organize my directories and packages?

The most frequent layout mistake in the corpus is adopting a structure that belongs to a different project type. The first branch is therefore the most important.

Tree#

Is this project intended to be imported as a library by external callers?
├── YES → Use flat root-package layout
│         Root package IS the product: cobra.Command{}, gin.New(), gorm.Open()
│         Sub-packages for specializations (binding/, render/, codec/)
│         Use internal/ only to hide hot-path utilities
│         Do NOT add cmd/ or pkg/ — they create verbose import paths
│         Exemplars: cobra, gin, echo, gorm, viper
│
└── NO (pure application binary or daemon)
    ├── Does it produce a single binary?
    │   ├── YES → Will it ever need to be imported as a library?
    │   │         ├── YES → Standard Go layout: cmd/<name>/main.go + internal/ + pkg/
    │   │         │         Use pkg/ only for the stable library surface
    │   │         │         Use internal/ for everything else
    │   │         │         Exemplars: helm (cmd/ + pkg/action), delve (cmd/ + pkg/proc)
    │   │         │
    │   │         └── NO  → All-internal layout: root main.go + everything under internal/
    │   │                   Compiler structurally enforces "not a library"
    │   │                   Exemplars: terraform, crush
    │   │                   Acceptable variant: root main.go + domain packages (no internal/)
    │   │                   When: team knows it's an app and wants max navigability
    │   │                   Exemplars: consul, nomad, vault, nats-server
    │   │
    │   └── NO (multiple binaries or services)
    │       ├── Are services deployed independently, need separate versioning?
    │       │   ├── YES → Multi-module monorepo with go.work
    │       │   │         Each client SDK or stable library gets its own go.mod
    │       │   │         Exemplars: etcd (13 modules), prometheus (5 modules)
    │       │   │         Cost: module graph management, CI complexity
    │       │   │
    │       │   └── NO  → Component sub-tree layout (single go.mod)
    │       │             Top-level dirs per service, each internally organized
    │       │             Shared code in common/ or pkg/
    │       │             Exemplars: dapr (6 services), temporal (4 services), istio
    │       │
    │       └── Is this a plugin-first system with many interchangeable backends?
    │           └── YES → Plugin-registry layout
    │                     Core abstraction in one package
    │                     Plugins in backends/<name>/ or modules/<name>/
    │                     Aggregator in backends/all/all.go (blank imports)
    │                     Exemplars: rclone (50+ backends), caddy (modules/)

What the corpus says about common mistakes#

The util package anti-pattern appears in at least eight projects (consul, vault, nomad, frp, buildkite-agent, and others). Packages named util/, helpers/, common/ become dumping grounds. When you find yourself creating one, the function belongs in an existing domain package, not a catch-all.

The god package — minio’s 453-file cmd/ package, nats-server’s 180-file server/ package — emerges from projects that grew quickly without planned decomposition. The short-term benefit (no inter-package API design) gives way to untestable code paths, slow compilation, and difficult onboarding. The signal that you’ve crossed the threshold: you can’t write a unit test for a function without initializing ten unrelated subsystems.

Premature pkg/ is declining for a reason. Of 51 projects, fewer than 10 use a pkg/ directory, and several use it reluctantly. If a package is public, give it a meaningful name, not a pkg/ prefix.

Decision 2: Concurrency Architecture#

The question: How should I manage goroutines, fan-out, and shutdown?

This decision has a clearer evolutionary path than any other. Projects are not right or wrong about their concurrency approach — they are early or late on the arc.

Tree#

Does your project spawn goroutines itself (not just the caller's problem)?
├── NO → You are a library. Spawn no goroutines.
│        GORM delegates connection pooling to database/sql; it spawns nothing.
│        If you must, make it opt-in with explicit lifecycle management.
│        Exemplars: cobra, gorm, viper, sqlc
│
└── YES → How many concurrent goroutines at peak?
    ├── < 10 goroutines (simple background work)
    │   Use signal.NotifyContext + sync.WaitGroup for the lifecycle
    │   Use go func() for background tasks, cancel via context
    │   Exemplars: air, pop, headscale, wireguard-go
    │
    ├── 10–50 goroutines (moderate fan-out)
    │   Use errgroup.WithContext(ctx) as the standard fan-out primitive
    │   Pattern: errgroup + input channel + N consumers = bounded worker pool
    │   Use signal.NotifyContext as the root lifecycle anchor
    │   Add errgroup.SetLimit(n) to bound concurrency without an explicit channel
    │   Exemplars: restic, rclone, gh, prometheus, hugo, drone
    │
    ├── 50–200 goroutines (high fan-out, production service)
    │   You need a goroutine registry.
    │   Minimum viable: WaitGroup-backed map that tracks every spawned goroutine
    │   Better: structured manager with quiesce/stop semantics
    │   Add goroutine leak detection to your test suite (goleak or leaktest)
    │   Is this a long-running daemon with subsystem independence requirements?
    │   ├── YES → Consider suture-style supervisor (syncthing's pattern)
    │   │         Each subsystem implements Serve(ctx) and gets restarted on failure
    │   └── NO  → CockroachDB-style Stopper or NATS-style startGoRoutine registry
    │             Central registry, ordered quiesce-then-stop
    │
    └── 200+ goroutines (infrastructure platform)
        Lifecycle management is non-negotiable — you are past the ad-hoc threshold
        Consider domain-specific pools:
        - Temporal's goro.Group + adaptive pool (auto-sizes on queue depth)
        - CockroachDB's sharded raftScheduler (lock-free at high QPS)
        - Kubernetes workqueue (level-triggered, with dedup and backoff)
        Add per-test goroutine leak detection (leaktest.AfterTest at 16,363 sites in CockroachDB)

The shutdown pattern decision#

Is your project started in 2019 or later?
├── YES → Use signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM)
│         This is the modern standard. The OS signal propagates as context cancellation.
│         Exemplars: prometheus, restic, rclone, buffalo, echo
│
└── NO  → Do you still use done-channel (chan struct{}) for shutdown?
    ├── YES → You have two cancellation systems if you also use context.Context.
    │         The bridging code accumulates silently.
    │         Plan a migration when the next forced refactor touches the startup path.
    │         The consul/vault pattern is a known liability, not a deliberate choice.
    └── NO  → Proceed; you are already on the modern path.

The sync.Pool decision#

Is there a hot path with a high-allocation short-lived object?
├── YES → Is the object's initialization cost high enough to justify reset complexity?
│         ├── YES → Use sync.Pool: pool.Get() → Reset() → use → pool.Put()
│         │         Exemplars: gin.Context, echo.context, fiber.DefaultCtx,
│         │                    nats-server (31 pool instances for Raft buffers)
│         └── NO  → Profile first. Premature pooling adds complexity without payoff.
└── NO  → Don't use sync.Pool.

Decision 3: Error Handling Strategy#

The question: How deeply should I invest in error handling infrastructure?

The corpus reveals a precise relationship between project scale and error strategy. Each upgrade is triggered by a specific event; adopting the next tier before that event arrives is premature.

Tree#

Who consumes your errors?
├── Humans (logs, terminal output)
│   Use stdlib sentinels and fmt.Errorf("%w", err)
│   Apply "failed to <verb> <object>: %w" convention consistently
│   DO use %w (not %v) — losing error chains is a silent bug
│   Exemplars: air, fzf, headscale, wireguard-go
│
└── Code (callers inspect errors for control flow)
    ├── Do callers only need to check identity? (errors.Is)
    │   ├── YES → Sentinel-dominant strategy
    │   │         Export named var Err* = errors.New("...") for each condition
    │   │         Wrap with fmt.Errorf("context: %w", ErrSpecific) to add detail
    │   │         Exemplars: prometheus, gorm, viper, cobra, nats-server
    │   │
    │   └── NO  → Do callers need structured data from errors? (errors.As)
    │       ├── YES → Rich custom type hierarchy
    │       │         Define error structs with typed fields per domain
    │       │         Use errors.As at handling sites to extract structured data
    │       │         Exemplars: moby (errdefs), gh, gitea, dapr, nomad, helm
    │       │
    │       └── Is this a multi-backend system with a centralized retry loop?
    │           ├── YES → Behavioral interface classification
    │           │         Errors carry behavioral metadata (Retrier, Fataler, NoRetrier)
    │           │         Central handler calls errors.Is/interface check, not type switch
    │           │         Exemplars: rclone (ShouldRetry), syncthing (FatalErr), restic (IsFatal)
    │           └── See protocol-boundary decision below
    │
└── Does your project serve an external API (HTTP, gRPC, CLI)?
    ├── YES → Add protocol-boundary translation at every handler boundary
    │         Internal error chains must NOT leak to external clients
    │         Pattern: errors.As/custom classifier → HTTP status or gRPC status
    │         Exemplars: minio (three-layer S3 translation), drone, dapr, etcd, traefik
    │
    └── Do errors cross a process or network boundary and need inspection on the other side?
        ├── YES → Structured errors with cross-process serialization
        │         Use proto-serializable error annotations
        │         Assign numeric error codes for wire stability
        │         Exemplars: CockroachDB (errors library + protobuf),
        │                    etcd (rpctypes.EtcdError), NATS (ApiError uint16 codes)
        └── NO  → Stay at the previous tier. Serialization overhead without need is waste.

The %w mandate#

One rule is not a decision — it is a mandate: always use fmt.Errorf("%w", err) not fmt.Errorf("%v", err) for wrapping. Buffalo uses %v throughout and has never migrated. The result is that errors.Is and errors.As are broken for all error chains in Buffalo’s codebase. consul, istio, and moby show %v in older paths — legacy debt, not a recommended practice. The cost of %w is zero. The cost of %v is silent breakage.

Decision 4: Extension and Plugin Mechanism#

The question: How do I let behavior be extended without modifying core code?

The plugin mechanism decision is the one most frequently gotten wrong by projects planning for future extensibility. The corpus reveals a clear selection criterion: the trust model.

Tree#

Who writes the plugins?
├── Your team (at compile time)
│   Use interface injection at construction
│   New() accepts conforming values; no registry, no subprocess
│   The extension point is a parameter, not a side-effecting call
│   Exemplars: gin (binding.Validator), echo (echo.Config{Router:...}),
│              gorm (db.Use(Plugin)), restic (backend decorator stack)
│
└── Third parties
    ├── Do plugins require recompilation of the host binary?
    │   ├── YES (acceptable) → init()-based self-registration
    │   │   Plugins call a global Register() from init()
    │   │   Host binary blank-imports all plugins (backend/all/all.go pattern)
    │   │   Trimmed builds omit specific plugins for smaller binaries
    │   │   Exemplars: rclone (50+ backends), caddy (xcaddy build tool),
    │   │              prometheus (service discovery providers),
    │   │              Kubernetes API types, GORM callbacks
    │   │
    │   └── NO (must load at runtime without recompile)
    │       ├── Is plugin code trusted (internal team, verified source)?
    │       │   ├── YES → Interpreted runtime
    │       │   │         Traefik + Yaegi: Go source at runtime, no compile step
    │       │   │         PocketBase + goja: JavaScript hooks for non-Go developers
    │       │   │         Cost: 5–20× slower than compiled; not for hot paths
    │       │   │
    │       │   └── NO  → Is language diversity or crash isolation required?
    │       │       ├── YES → Subprocess + gRPC (hashicorp/go-plugin)
    │       │       │         Plugin is a separate process, mTLS gRPC communication
    │       │       │         Plugin crash cannot bring down the host
    │       │       │         Language-agnostic: any language that speaks gRPC
    │       │       │         Exemplars: vault (logical.Backend),
    │       │       │                    terraform (Provider proto),
    │       │       │                    grafana (pluginv2), nomad (DriverPlugin)
    │       │       │         Cost: ~5ms launch, ~1ms round-trip; not for request hot paths
    │       │       │
    │       │       └── Is the extension point request-scoped?
    │       │           └── YES → Middleware chains
    │       │                     func(next Handler) Handler is sufficient
    │       │                     Zero subprocess overhead, zero registry
    │       │                     Exemplars: every HTTP framework in the corpus
    │       │
    │       └── Do plugins need to be deployed as independent processes (microservices as plugins)?
    │           └── YES → Language-agnostic gRPC protocol sockets
    │                     Plugins listen on Unix sockets; host discovers and connects
    │                     Exemplars: Kubernetes (CRI, CSI, Device Plugin), Dapr
    │                     Cost: operational complexity per plugin process

Decision 5: Dependency Injection Approach#

The question: How should I wire my application’s dependencies?

The corpus’s clearest finding on this topic: 75% of projects wire dependencies by hand, and this is the right answer for 75% of projects. The question is not “should I use DI?” — Go always does DI. The question is when manual wiring stops being feasible.

Tree#

Does the dependency graph fit in one person's working memory?
├── YES → Manual constructor injection at main() or NewApp()
│         NewServer(db, logger, cache) is the Go standard
│         The composition root is the transparent record of all wiring decisions
│         Exemplars: virtually every Tier S/M project in the corpus
│
└── NO  → Is the graph too large to hand-maintain error-free?
    ├── Defining threshold: ~1000 lines in the composition root, or
    │   multiple services with dozens of components each
    │
    ├── Do you want generated wiring that is human-readable and greppable?
    │   └── YES → Google Wire
    │             Generates a concrete wire_gen.go file
    │             Checked in, auditable, no runtime magic
    │             Exemplar: Grafana (1939 lines of generated wiring across 35 modules)
    │
    └── Do you have multiple independently deployable services, each with their own graph?
        └── YES → uber/fx with nested fx.App instances
                  More flexible, resolved at runtime
                  Harder to audit than Wire but handles dynamic graphs better
                  Exemplar: Temporal (4 services, each with separate fx.App)

The key insight: A large composition root is a feature, not a code smell. Prometheus’s cmd/prometheus/main.go at 1700 lines and Caddy’s initialization path are both large because their dependency graphs are large. The size is the faithful record of the wiring decisions. The only problem with a large composition root is when it becomes too large to maintain confidently — that is the threshold where Wire or fx add value.

Decision 6: API Surface Design#

The question: How many API surfaces should my project expose, and how should they relate?

The corpus reveals a near-deterministic relationship between project type and surface count. Surface multiplication is not a design choice so much as an outcome of how many different consumer categories the project must serve.

Tree#

How many distinct consumer categories does your project serve?
├── 1 (only library callers, or only terminal users)
│   One surface. No REST, no gRPC, no plugin protocol.
│   Exemplars: cobra (library), fzf (terminal), air (developer CLI)
│
├── 2 (developers + automation/operators)
│   One primary surface + one management/query surface
│   Pattern: CLI for humans + REST for automation
│   Exemplars: rclone (CLI + rcd REST), syncthing (daemon + web UI), headscale (CLI + REST)
│
├── 3 (developers + operators + embedding)
│   Three surfaces: typically CLI + REST API + library/embedding path
│   The third surface almost always appears when someone wants to embed the tool in tests
│   or in a CI pipeline without subprocess overhead
│   Exemplars: prometheus (CLI + REST + library), caddy (CLI + admin REST + module API),
│              traefik (CLI + dynamic config API + proxy surface)
│
└── 4+ (platform serving developers, operators, automation, and third-party integrators)
    Each surface needs: its own router, its own auth model,
    its own error mapping, and its own versioning strategy
    Gitea's isolation rule: no surface's handler code imports another surface's context type
    Exemplars: Gitea (REST + 20+ package registry protocols + web UI + Git SSH),
               Kubernetes (kubectl + apiserver REST + internal gRPC + CRI + CNI + CSI + webhooks),
               Temporal (WorkflowService gRPC + internal gRPC + REST + Nexus HTTP + CLI)

The isolation imperative for multi-surface systems#

When surfaces multiply, the failure mode is cross-surface coupling: a handler for the REST API imports the context type from the web UI handler, which imports the git handler’s auth model. The technical debt is invisible until it prevents independent evolution of each surface.

The correct architecture at 3+ surfaces: each surface is a separate router/handler package. No surface imports another surface’s context types. Error mapping at each surface boundary is independent. Gitea is the clearest exemplar of this discipline applied at scale. The moby errdefs package — twelve behavioral error interfaces, each mapping to a specific HTTP status code — is the clean model for how internal errors translate to external codes without surface coupling.

Decision 7: Testing Strategy#

The question: How much should I invest in test infrastructure, and what kind?

Testing is the dimension most tightly correlated with tier, because the cost of comprehensive testing scales superlinearly. The wrong investment at the wrong tier is as harmful as under-investment.

Tree#

What is your project tier?
├── Tier S (< 50 files, 1–2 developers)
│   Co-located _test.go files, stdlib-first
│   Table-driven tests with t.Run(tc.name, ...) for all non-trivial functions
│   No test infrastructure investment yet
│   Exemplars: cobra (46 table-driven occurrences), wireguard-go, air
│
├── Tier M (50–500 files, small team)
│   Table-driven tests + testify/assert and testify/require
│   testify is the most universal dependency in the corpus (47 of 51 projects)
│   The test suite should run in < 30 seconds for developer feedback
│   Consider: does your project start goroutines that could leak?
│   ├── YES → Add goleak.VerifyTestMain(m) NOW, before goroutine leak debt accumulates
│   └── NO  → Skip goroutine leak detection until you spawn goroutines
│   Exemplars: restic, rclone, crush, headscale, syncthing
│
├── Tier L (500–2000 files, 10–30 contributors)
│   Two strategies diverge here — both valid:
│   ├── Integration-first (no mocks):
│   │   Start real server instances in-process; refuse substitutes
│   │   Zero mock drift, zero mock maintenance burden
│   │   Exemplars: nats-server (real 3-node clusters), pocketbase (real SQLite app),
│   │              minio (real object stores), caddy, fyne
│   │
│   └── Build-tag separation:
│       //go:build integration separates slow from fast tests
│       Unit tier uses interface-injected fakes; runs in milliseconds
│       Integration tier runs on CI with real infrastructure
│       Exemplars: prometheus, grafana, gitea
│   │
│   Either way: add goroutine leak detection if you spawn goroutines
│
└── Tier XL (2000+ files, 30+ contributors)
    Three-tier pyramid: unit → integration → E2E (separated by build tags or directories)
    Invest in domain-specific test DSLs for your primary test scenario type:
    - CockroachDB: logictest (493 SQL files × 8 config variants)
    - Prometheus: promqltest (PromQL query vectors)
    - Kubernetes: envtest + integration suites
    - Caddy: .caddyfiletest files
    - PocketBase: ApiScenario struct (200+ test scenarios in a DSL)
    
    A domain test DSL gives dramatically more test cases per line of test code.
    If your project has a primary language or protocol (SQL, PromQL, HTTP config),
    build the DSL for it. The upfront cost is paid back within the first 100 test cases.
    
    Add per-test goroutine leak detection (leaktest.AfterTest or equivalent)
    Exemplars: cockroach (16,363 leak-detection sites), prometheus (goleak in 30 packages)

The VCR cassette decision#

Does your project test against non-deterministic external APIs
(LLM inference, third-party REST APIs, external data sources)?
├── YES → VCR cassette replay
│         Record real conversations to testdata/ cassette files
│         CI replays them — deterministic, fast, free
│         Exemplar: crush (charm.land/x/vcr for LLM conversations, with -race flag)
│         For any AI agent or third-party API client: this is the only viable strategy
│         Live calls are flaky and expensive; pure mocks lose real conversation dynamics
└── NO  → Skip.

Putting It Together: The Forcing Functions#

The seven trees above describe independent dimensions. The critical insight from the corpus is that each dimension has an independent forcing function — the specific event that makes the current approach insufficient.

Watching for these forcing functions — and acting when they arrive rather than deferring — is what separates teams that scale gracefully from those that accumulate architectural debt.

Cross-Cutting Rules That Override Everything#

Three rules from the corpus apply regardless of which branch you are on:

1. fmt.Errorf("%w") always. Using %v for error wrapping breaks errors.Is and errors.As silently. The cost is zero. The debt is invisible and expensive. Buffalo is the cautionary example in the corpus of what happens when a project standardizes on %v and never migrates.

2. context.Context as the first parameter on every blocking function. The three outliers — Gin (predates and disagrees), fzf (pure algorithm), fyne (GUI event loop) — each represent a principled exception. Every other project treats this as non-negotiable. Starting context-first gives you a cancellation path from the outermost entry point down through every blocking operation. Adding it retroactively to consul, vault, and the older Kubernetes controller paths has taken years and is still incomplete.

3. Table-driven tests with t.Run(tc.name, ...) from the first test file. The pattern appears in all 51 projects. The t.Run wrapper enables individual case selection (go test -run TestFoo/my_case_name), which saves debugging time when specific combinations fail. The cost is zero. Starting without it and retrofitting later is non-trivial at scale.

The Master Decision Sequence for a New Project#

When starting a new Go project, apply these decisions in order:

1. Library or application? This determines layout before any other question.
2. Single consumer category or multiple? This determines API surface count.
3. Goroutine count estimate? Start with signal.NotifyContext + errgroup. Plan for a registry if you’ll cross 50 goroutines.
4. Error consumer: human or code? Start with stdlib sentinels. Upgrade when callers first need errors.As.
5. Extension: your team or third parties? Interface injection for your team. Plan the subprocess boundary before you have third-party authors, not after.
6. Test infrastructure: invest in goroutine leak detection early. The incremental cost is low; the debt from not having it accumulates silently.

The projects that scale best in this corpus — Temporal, etcd, restic, rclone, Caddy — all made the right call at each of these decision points at roughly the moment they needed to, not a year before (over-engineering) and not a year after (debt). The corpus shows what “roughly the right moment” looks like in practice. Use it.

Note on fyne and crush#

fyne sits outside most of these trees. Its concurrency model (main-thread marshaling via fyne.Do(), no errgroup, no context) reflects GUI domain conventions that apply universally to GUI toolkits and nowhere else. Apply these trees to fyne only for error handling and layout decisions; the concurrency, DI, and API surface trees do not apply.

crush is at the leading edge of several trees simultaneously: all-internal layout (Layout Decision 1), errgroup + signal.NotifyContext (Concurrency Decision), interface injection + init() for extensions, VCR cassette replay for LLM testing (Testing Decision). It represents the decisions a 2025 project makes when starting from current best practices with no legacy constraints. The trees above should produce approximately crush’s choices for a new application of similar scope.