Harness Open Source (Drone/Gitness) — Patterns#

Concurrency patterns#

errgroup-based structured concurrency#

  • Usage: Main server startup and parallel git API calls.
  • Example: cli/operations/server/server.go:85 — the entire server (HTTP, SSH, metrics, poller, job scheduler) is launched with errgroup.WithContext. git/api/commit.go:243 uses errgroup to fan-out blame/diff operations over many files in parallel.
  • Assessment: Idiomatic and effective. All goroutines share a parent context; first failure cancels the group. The pattern prevents orphaned goroutines at startup/shutdown.

Worker pool (pipeline scheduler)#

  • Usage: app/pipeline/scheduler/queue.go maintains a map[*worker]struct{} of active polling workers. When a new stage is queued, the scheduler iterates workers and wakes matching ones via a select/channel signal.
  • Example: app/pipeline/scheduler/queue.go:38workers map[*worker]struct{} is the live-set of connected drone-runner clients; config.CI.ParallelWorkers is the concurrency cap for the in-process runner (app/pipeline/runner/runner.go:102).
  • Assessment: Effective for the polling/long-poll model. Each runner registers itself as a worker, and the scheduler does O(n-workers) matching on stage dispatch.

Semaphore pattern (channel-based)#

  • Usage: S3 and filesystem storage drivers use make(chan struct{}, concurrency) as a semaphore for multipart upload concurrency limits.
  • Example: registry/app/driver/s3-aws/s3.go:264limiter := make(chan struct{}, d.MultipartCopyMaxConcurrency).
  • Assessment: Classic, idiomatic Go. Buffer size acts as the concurrency limit; goroutines block on send until a slot is free.

Context cancellation throughout#

  • Usage: 6,807 occurrences of context.Context across the codebase. Context is threaded through every operation — HTTP handlers, git commands (git/command/), SQL queries, event handlers.
  • Example: git/command/command.go wraps all exec.Cmd calls with context; cancellation propagates to the OS subprocess.
  • Assessment: Excellent. Context use is pervasive and idiomatic — every blocking call can be cancelled. The graceful shutdown sequence (server.go:160) creates a bounded timeout context for draining.

Graceful shutdown#

  • Usage: cli/operations/server/server.go:46 uses signal.NotifyContext for SIGINT/SIGTERM. When triggered, a context.WithTimeout(config.GracefulShutdownTime) drains servers in order: HTTP → SSH → metrics → instrumentation → job scheduler.
  • Example: server.go:157-180 — explicit, ordered shutdown sequence with logged errors at each step.
  • Assessment: Correct and production-quality. The timeout prevents indefinite drain.

Publish/subscribe fan-out (event bus)#

  • Usage: The stream/ package provides an in-memory broker where multiple consumer groups receive each message independently. Redis Streams provide the same model in production. Background services subscribe via events.ReaderFactory.Launch().
  • Example: stream/memory_broker.go:39messageQueues map[string]map[string]chan message maps streamID → groupName → message channel. Messages are fanned out to all groups.
  • Assessment: Clean abstraction. The dual-mode (in-memory for testing/single-node, Redis for production) makes the event bus both testable and production-ready.

select {} as timeout/cancellation guard#

  • Usage: 74 select {} blocks, primarily in event readers, job scheduler loops, and SSH session handlers.
  • Assessment: Standard Go idiom for multiplexing on context cancellation and data arrival.

Error handling#

  • Style: Domain-specific typed error with HTTP status code mapping. The project uses a custom errors.Error struct (in the errors/ package) with a Status field (not_found, conflict, unauthorized, etc.) rather than plain sentinel errors or pkg/errors.
  • Error types defined:
    • errors.Error (errors/status.go) — primary application error type with Status, Message, Err, and Details fields.
    • errors.Status — string enum for machine-readable codes (StatusConflict, StatusNotFound, StatusUnauthorized, etc.).
    • client.remoteError — wraps HTTP error responses from the gitness REST client.
  • Wrapping approach: fmt.Errorf("%w", err) throughout callsites for wrapping stdlib/third-party errors. The custom errors.Error.Unwrap() correctly participates in the errors.Is/errors.As chain.
  • Generics in error handling: errors.IsType[T error](err error) bool (errors/util.go:19) uses a generic type parameter to check if an error unwraps to a specific type without a type assertion — a concise, type-safe pattern.
  • HTTP mapping: HTTP handlers map errors.AsStatus(err) to HTTP status codes in a central location, keeping the controller layer free of HTTP knowledge.
  • Examples:
    • errors/status.go:44Error.Error() formats as "<message>: <wrapped error>".
    • cli/operations/server/config.go:72 — all config errors are wrapped with context via fmt.Errorf("...: %w", err).
    • cli/session/session.go:26 — sentinel: ErrTokenExpired = errors.New("token is expired, please login") for CLI-layer errors.

Configuration pattern#

  • Approach: Config struct injected via Google Wire. Sub-configs are narrow slices extracted by Provide*Config helper functions.
  • Example: The top-level types.Config has nested sub-structs (Config.HTTP, Config.Database, Config.CI, etc.). cli/provide/ contains functions like ProvideGitConfig(config *types.Config) *git.Config that extract just the git-relevant sub-struct, allowing each package to receive only its own config. No global config object — each injected type is a *domain.Config.
  • Assessment: Excellent isolation. Packages cannot accidentally read each other’s config. Adding a new field requires updating only the provider function and the domain config struct.

Dependency injection#

  • Approach: Google Wire, compile-time code generation.
  • Evidence: app/router/wire.go defines the Wire ProviderSet for the router; cmd/gitness/wire_gen.go is the generated constructor chain (~2000 lines) that instantiates all ~200 objects in dependency order.
  • Pattern: Each package exposes a wire.go with a WireSet (a wire.ProviderSet) listing its constructor and provider functions. The root wire.go aggregates ~120 WireSets in a single wire.Build(). There is no runtime container — the generated code is plain Go function calls.
  • Significance: At this scale (120 WireSets, 200+ objects), compile-time DI catches missing constructors at build time rather than panicking at startup. This is a deliberate engineering trade-off: verbosity at configuration time, reliability at runtime.

Other notable patterns#

Functional options (FuncOption + Option interface)#

  • The audit/ package defines a clean FuncOption func(e *Event) type implementing an Option interface with an Apply(*Event) method. Options are WithID, WithNewObject, WithOldObject, WithClientIP, WithData, etc.
  • audit/audit.go:207-247 — the FuncOption approach (function type that implements an interface) is the idiomatic dual-pronged pattern that allows both closures and concrete types to satisfy Option.
  • Used in moderation: only where events have optional, contextual fields. The rest of the codebase uses plain struct constructors injected via Wire.

Generics for type-safe event handling#

  • events/events.go:28Event[T any] is the core generic event envelope.
  • events/reader.go:146HandlerFunc[T any] func(context.Context, *Event[T]) error is the typed handler signature.
  • events/reader.go:160ReaderRegisterEvent[T any](reader *GenericReader, ...) is a package-level generic function (not a method) because Go 1.18+ does not allow generic methods on non-generic types. This is a real constraint workaround, documented in the comment.
  • git/stream.go:20StreamReader[T any] wraps a channel pair for async streaming results.
  • Assessment: Generics are used precisely where they provide real value: eliminating interface{} + type assertion in the event bus hot path. The workaround for the “no generic methods” limitation is well-documented.

Context as value store (scoped to middleware)#

  • audit/context.go and audit/middleware.go use context.WithValue to propagate HTTP request metadata (IP, path, method, request ID) to audit event builders downstream.
  • ssh/middleware.go:133 reads a zerolog.Logger from context using a typed key.
  • Assessment: Used narrowly and correctly — only for cross-cutting concerns (observability, audit) that are genuinely request-scoped, not for passing business data.

Table-driven tests#

  • Prevalence: Heavy — 248 matches for testCases/tt.Run/tc.name patterns across test files.
  • Style: Anonymous struct slice ([]struct{ name string; ... }) iterated with t.Run(tc.name, ...).
  • Framework: testify/assert and testify/require throughout; testify/mock for generated mocks in the registry sub-module.

Builder-style via chaining on error types#

  • errors.Error supports SetErr(err) and SetDetails(details) returning *Error, allowing fluent construction: &errors.Error{Status: ..., Message: ...}.SetErr(originalErr).

Observer / event system#

  • Services implement a Register(ctx context.Context) error lifecycle method that subscribes to Redis Stream consumer groups via events.ReaderFactory.Launch(). The Register method launches goroutines internally — the caller only has to invoke Register once during startup.
  • This is effectively an observer pattern where services self-subscribe to domain events rather than being explicitly wired as listeners.

Registry pattern (CLI commands)#

  • Each CLI sub-package (server, migrate, users, account) exposes a Register(app *kingpin.Application) function. cmd/gitness/main.go calls each one to build the command tree. This avoids a central command list — adding a new command only requires a Register call in main.go.

In-memory vs Redis dual-mode broker#

  • stream/ provides both a MemoryBroker and a Redis-backed broker implementing the same Broker interface. The event and pubsub systems select mode via config (ModeRedis / ModeInMemory). This allows full in-process testing without Redis and production deployment with Redis — a pattern worth emulating for any system that requires a message broker.