Harness Open Source (Drone/Gitness) — Architecture#

Architectural style#

Layered Monolith with Event-Driven Cross-domain Integration

The core is a traditional three-tier layered monolith (handler → controller → service → store) compiled to a single binary, but cross-domain workflows (e.g., git push triggering CI, repository events triggering webhooks) use a Redis Streams event bus rather than direct in-process calls. This hybrid produces a system that is operationally simple (one binary, one database) but architecturally decoupled at the domain boundary.

Evidence: The app/router/wire.go assembles four independent sub-routers (Git, Registry, API, Web). The wire.go lists ~120 WireSets in a single wire.Build(). Event publishers (app/events/git/, app/events/pullreq/, app/events/pipeline/) are injected into controllers; event subscribers live in app/services/trigger/, app/services/webhook/, etc.

The registry/ sub-module is separately versioned but compiled into the same binary — a modular sub-monolith.

Component diagram (textual)#

┌──────────────────────────────────────────────────────────────┐
│                     Single Gitness Binary                    │
│                                                              │
│  ┌─────────┐  ┌──────────┐  ┌────────────┐  ┌───────────┐  │
│  │  Git    │  │ Registry │  │   REST API │  │  Web (SPA)│  │
│  │ Router  │  │  Router  │  │   Router   │  │  Router   │  │
│  └────┬────┘  └────┬─────┘  └─────┬──────┘  └─────┬─────┘  │
│       │            │              │                │         │
│  ┌────▼────┐  ┌────▼─────┐  ┌────▼──────────────────────┐  │
│  │ git/    │  │registry/ │  │  app/api/controller/       │  │
│  │ (native │  │ app/     │  │  (30+ domain controllers)  │  │
│  │  git ops│  │ (OCI     │  └────────────┬───────────────┘  │
│  │  + LFS) │  │ registry)│               │                  │
│  └────┬────┘  └────┬─────┘  ┌────────────▼───────────────┐  │
│       │            │        │  app/services/              │  │
│       │            │        │  (40+ domain services)      │  │
│       │            │        └────────────┬───────────────┘  │
│       │            │                     │                  │
│       │            │        ┌────────────▼───────────────┐  │
│       │            │        │  app/store/ (interfaces)   │  │
│       │            │        │  + database/ (SQL impls)   │  │
│       │            │        └────────────────────────────┘  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  Shared Infrastructure Layer (root-level packages)   │   │
│  │  events/ │ pubsub/ │ livelog/ │ job/ │ lock/ │ blob/ │   │
│  │  ssh/    │ encrypt/│ cache/   │ git/ │ audit │ stream│   │
│  └──────────────────────────────────────────────────────┘   │
│                                                              │
│  ┌───────────────┐  ┌─────────────────────────────────────┐ │
│  │ app/pipeline/ │  │ app/gitspace/                       │ │
│  │ (CI engine;   │  │ (Cloud Dev Environments;            │ │
│  │  drone runner │  │  Docker infra + IDE orchestration)  │ │
│  │  integration) │  └─────────────────────────────────────┘ │
│  └───────────────┘                                           │
│                                                              │
│  External: Redis (events/pubsub/livelog/lock/cache)          │
│            SQLite / PostgreSQL (primary data store)          │
│            Docker daemon (pipeline runner + gitspaces)       │
└──────────────────────────────────────────────────────────────┘

  SSH Server (git clone/push over SSH)
  ↕ gliderlabs/ssh

  Drone Runner (separate process, polls manager API)
  ↕ drone/runner-go poller

Core components#

HTTP Router (app/router/)#

  • Package: github.com/harness/gitness/app/router
  • Responsibility: Dispatches incoming HTTP requests to one of four sub-routers based on URL prefix and optionally Host header. The top-level Router.ServeHTTP iterates a slice of Interface implementations calling IsEligibleTraffic for each.
  • Key types: Router (outer dispatcher), Interface (eligibility + handler contract), APIRouter, GitRouter, WebRouter, plus router.AppRouter from the registry sub-module.
  • Dependencies: authn.Authenticator, all domain controllers (injected), git.Interface, openapi.Service, url.Provider

API Handler + Controllers (app/api/)#

  • Package: github.com/harness/gitness/app/api/{handler,controller}/
  • Responsibility: Thin HTTP handlers decode requests and delegate to controllers. Controllers are the business logic layer: they enforce authorization, validate input, orchestrate services, and return domain objects. There are 30+ domain-specific controller packages (repo, space, pullreq, pipeline, execution, gitspace, etc.).
  • Key types: Each domain has a Controller struct with injected dependencies. Example: repo.Controller takes store.RepoStore, git.Interface, event publishers, authz.Authorizer, url.Provider, etc.
  • Dependencies: app/services/, app/store/, git/, events/, authz.Authorizer, authn.Authenticator

Domain Services (app/services/)#

  • Package: github.com/harness/gitness/app/services/<domain>/
  • Responsibility: Cross-controller logic and background event processing. Services subscribe to domain events (via events.ReaderFactory) and perform async work: webhook delivery, pull request state management, pipeline triggering, notification dispatch, language analysis, keyword indexing. The Services aggregate struct in app/services/wire.go is the top-level DI node for all services.
  • Key types: webhook.Service, pullreq.Service, trigger.Service, webhook.Service, cleanup.Service, notification.Service — each implements Register(ctx) error to subscribe to event streams and launch goroutines.
  • Dependencies: events.ReaderFactory, job.Scheduler, app/store/, git.Interface

Data Store (app/store/)#

  • Package: github.com/harness/gitness/app/store/ (interfaces), app/store/database/ (SQL implementations), app/store/cache/ (Redis cache layer), app/store/logs/ (log storage)
  • Responsibility: All persistence is behind interfaces defined in app/store/. SQL implementations use sqlx (raw SQL, no ORM). Cache implementations wrap the SQL stores with Redis-backed TTL caches for hot paths (space/repo lookups). Database migrations use dbmate.
  • Key types: store.RepoStore, store.SpaceStore, store.PipelineStore, store.ExecutionStore, store.PrincipalStore — all interfaces in app/store/. Concrete types: database.RepoStore, cache.SpaceCache, etc.
  • Dependencies: sqlx, redis/v9, dbtx (transaction helper in store/database/dbtx/)

Event Bus (events/)#

  • Package: github.com/harness/gitness/events/
  • Responsibility: A typed, persistent event bus backed by Redis Streams. Provides ReaderFactory[R] (consumer groups) and Reporter (publisher) generics. Domain-specific event packages in app/events/<domain>/ define typed event payloads and wrap the generic factory.
  • Key types: ReaderFactory[R], GenericReader, ReaderCanceler, StreamConsumerFactoryFunc. Domain wrappers: gitevents.Reporter, pullreqevents.Reporter, pipelineevents.Reporter.
  • Dependencies: go-redis/redis for Redis Streams, Go encoding/gob for payload serialization

Pipeline Subsystem (app/pipeline/)#

  • Package: github.com/harness/gitness/app/pipeline/{manager,scheduler,triggerer,runner,converter,file,canceler,commit}/
  • Responsibility: CI/CD execution engine inherited from Drone. The manager package implements ExecutionManager, which is the RPC-over-HTTP interface that external drone-runner-docker processes poll. The scheduler assigns pending stages to available runners. The triggerer evaluates pipeline trigger conditions. The converter translates Drone YAML to the Harness execution spec. The runner package runs an in-process poller (using drone/runner-go/poller).
  • Key types: manager.Manager (implements ExecutionManager), scheduler.Scheduler (wraps a DB-backed queue), triggerer.Triggerer, runner.Runner, converter.Converter
  • Dependencies: drone/runner-go, drone/drone-yaml, external drone runners via HTTP poll

Git Engine (git/)#

  • Package: github.com/harness/gitness/git/
  • Responsibility: Native git operations without Gitea/Gogs. Wraps the system git binary via git/command/ (exec-based). Provides high-level operations in git/api/: commits, refs, blobs, diffs, merges, blame. The git.Interface is the primary abstraction consumed by the repo controller.
  • Key types: git.Interface, git.Repository, api.Client (the internal adapter), command.Command (low-level exec wrapper)
  • Dependencies: system git binary (exec), git/storage/ (on-disk repository layout)

SSH Server (ssh/)#

  • Package: github.com/harness/gitness/ssh/
  • Responsibility: Provides SSH git clone/push/fetch access using gliderlabs/ssh. Authenticates via public key (stored in app/store/). Routes git-protocol requests to the same git engine used by HTTP.
  • Key types: ssh.Server
  • Dependencies: gliderlabs/ssh, authn.Authenticator, git/

OCI Registry (registry/)#

  • Package: github.com/harness/gitness/registry/ (separate Go module)
  • Responsibility: Docker Hub–compatible artifact registry (OCI distribution spec). Handles image push/pull, garbage collection, artifact replication, and webhook delivery for registry events. Declared as a workspace sub-module with its own go.mod but compiled into the same binary via a replace directive.
  • Key types: registry/app/api/ (HTTP handlers), registry/services/ (business logic), registry/gc/ (garbage collector), registry/job/ (background jobs)
  • Dependencies: Shares events/, store/database/dbtx/, types/ from the root module

Gitspace Subsystem (app/gitspace/)#

  • Package: github.com/harness/gitness/app/gitspace/{infrastructure,orchestrator,scm,platformconnector,secret}/
  • Responsibility: Cloud Development Environment management. Provisions Docker containers as remote dev environments, injects IDE tooling (VS Code Web, Cursor, JetBrains), and manages the lifecycle via an orchestrator. Infrastructure provisioning is abstracted through infraprovider.InfraProvider.
  • Key types: orchestrator.Orchestrator, infrastructure.InfraProvisioner, containerorchestrator.Orchestrator, ide.IDE
  • Dependencies: infraprovider/ (Docker), app/events/gitspace*/, git/

Data flow#

Git push over HTTP (triggers CI)#

Client HTTP push
  → GitRouter.IsEligibleTraffic (prefix = repo path, not /api/)
  → NewGitHandler (chi router) → authn middleware
  → handlerrepo.HandleGitInfoRefs / HandleGitUploadPack
  → repo.Controller.GitServicePack
  → git.Interface.CreateCommit / UpdateRef
  → git hook fires (server-side, pre/post-receive)
  → app/githook handler called (HTTP back to gitness)
  → githook.Controller.PostReceive
  → gitevents.Reporter.BranchUpdated / TagCreated (→ Redis Stream)
  ← event published asynchronously

[Background, concurrently]
  trigger.Service (subscriber via events.ReaderFactory)
  → reads BranchUpdated from Redis Stream
  → triggerer.Triggerer.Trigger
  → loads .drone.yml / pipeline YAML from repo
  → converter.Converter translates YAML
  → execution created in DB
  → scheduler.Scheduler.Schedule (inserts stage into queue)

[External drone-runner-docker process]
  → manager HTTP poll: GET /rpc/v2/stage (long-poll)
  → manager.Manager.Request: dequeue stage from scheduler
  → runner executes Docker containers per step
  → manager.Manager.Log (→ livelog Redis pub/sub → SSE to browser)
  → manager.Manager.AfterStep / AfterStage
  → execution updated in DB
  → pipelineevents.Reporter.ExecutionUpdated (→ Redis Stream)
  → commit status posted to SCM

REST API request (e.g., create pull request)#

Client HTTP POST /api/v1/repos/{repo_ref}/pullreq
  → APIRouter.IsEligibleTraffic (prefix = /api/)
  → NewAPIHandler (chi router)
  → authn middleware (JWT/token → Principal)
  → handler/pullreq.HandleCreate
  → pullreq.Controller.Create
  → authz.Authorizer.Check (permission check)
  → app/store.PullReqStore.Create (SQL INSERT via sqlx)
  → pullreqevents.Reporter.Created (→ Redis Stream)
  ← 201 response to client

[Background]
  webhook.Service (subscriber)
  → reads PullReqCreated from Redis Stream
  → fetches registered webhooks from store
  → HTTP delivery to configured endpoints
  notification.Service (subscriber)
  → sends email notifications

Initialization / Bootstrap#

Startup sequence (sequential):

  1. main() — registers kingpin CLI commands; server.Register stores initSystem as the wire initializer
  2. command.run() — loads .env file via godotenv, calls LoadConfig() (pure env var processing via kelseyhightower/envconfig)
  3. initSystem(ctx, config) — calls the Wire-generated wire_gen.go which constructs ~200+ objects in dependency order via explicit constructor calls
  4. system.bootstrap(ctx) — creates the admin user and system service principals (pipeline, gitspace) in the database if they don’t exist; idempotent on restart
  5. errgroup launches background goroutines:
    • services.JobScheduler.Run(gCtx) — starts the background job scheduler loop plus registered jobs (cleanup, metric collection, repo size calculation)
    • system.server.ListenAndServe() — starts the HTTP server (plain, TLS, or ACME depending on config)
    • system.metricServer.ListenAndServe() — optional metrics endpoint (no-op in OSS build)
    • system.sshServer.ListenAndServe() — conditional on config.SSH.Enable
    • system.resolverManager.Populate(ctx) — pre-fetches plugin metadata (if CI enabled)
    • system.poller.Poll(ctx, config.CI.ParallelWorkers) — starts the in-process Drone runner poller (if CI enabled)
  6. <-gCtx.Done() — blocks until OS signal or error
  7. Graceful shutdown: HTTP server → SSH server → metric server → instrumentation → job scheduler drain

Dependency injection: Google Wire, compile-time. The wire.go file contains a single wire.Build() with ~120 WireSet references. wire_gen.go is the machine-generated constructor chain that is actually compiled. No runtime DI container; if the graph is incomplete, the build fails.

Services start themselves: Each domain service in app/services/ implements a Register(ctx) error method that calls events.ReaderFactory.Launch() to subscribe to Redis Streams. These are called from the job scheduler goroutine or directly in the errgroup.

Configuration#

  • Mechanism: Pure environment variables, processed by kelseyhightower/envconfig
  • Config struct: types.Config — a flat struct with embedded sub-structs for each domain (Config.HTTP, Config.SSH, Config.Database, Config.Redis, Config.Git, Config.CI, Config.Gitspace, etc.)
  • Loading: LoadConfig() in cli/operations/server/config.go calls envconfig.Process("", config) then derives computed values (URL backfilling, instance ID from hostname)
  • .env file: Optional; loaded via godotenv before config processing. Useful for local development (make run uses .local.env).
  • No Viper, no YAML config files. The design is deliberately 12-factor: all config comes from the environment, making it container-native from day one.
  • Config propagation: *types.Config is the single config node injected into Wire; sub-configs are extracted via Provide*Config helper functions (e.g., cliserver.ProvideGitConfig, cliserver.ProvideLockConfig) which narrow the config to just the fields a package needs.

Key design decisions#

  1. Event bus as the domain boundary. Rather than direct service-to-service calls between domains (git → CI, PR → webhooks), the platform publishes events to Redis Streams and subscribes in separate goroutines. This prevents tight coupling between git hosting and CI execution — the git subsystem doesn’t know CI exists. It also enables future distribution (move the trigger service out-of-process) without API changes.

  2. Four-router HTTP dispatch pattern. The outer Router is a slice of Interface implementations checked in priority order (GitRouter → RegistryRouter → APIRouter → WebRouter). This avoids a single giant chi mux with conflicting patterns between git’s raw path routing and the REST API’s /api/v1/ prefix. Each sub-router owns its traffic classification and prefix stripping independently.

  3. Google Wire at scale (single binary with 120 WireSets). The DI graph is compile-time verified and explicit. There is no interface{} registry or runtime resolution. The cost is the unwieldy wire.go file and re-running wire on every new dependency. The benefit is that “missing constructor” becomes a build error, not a runtime panic — critical for a system this complex.

  4. Registry as a workspace sub-module. registry/ has github.com/harness/gitness/registry as its module path and its own go.mod, connected via replace github.com/harness/gitness/registry => ./registry in the root module. This gives the registry team API evolution independence while still compiling into the same binary. It mirrors how Google’s monorepo teams use Go modules within a single repository.

  5. Drone runner as an external process, not embedded. Despite being a CI platform, the actual build execution (container scheduling, step execution, log streaming) is handled by a separate drone-runner-docker process that polls the manager.Manager RPC API. This keeps the gitness server stateless with respect to container lifecycle and allows the runner to scale independently. The in-process poller (app/pipeline/runner/) runs a minimal drone runner for single-node setups.