Harness Open Source (Drone/Gitness) — Testing#

Test metrics#

  • Test files: 212 *_test.go files (excluding vendor)
  • Total Go files: 2,578
  • Ratio (test files / source files): ~1:12 (~8%) — sparse for a project of this size
  • Unique packages with tests: 98
  • Table-driven test occurrences: 577 (testCases, t.Run, tt.Run patterns)
  • Test frameworks: testify/assert, testify/require (primary); testify/mock (mocking); mockery v2.53.3 (code generation); stdlib testing (some packages)

Test organization#

  • Placement: Mixed — some tests are in the same package (white-box, e.g., audit/context_test.go in package audit), others use the external _test package (black-box, e.g., registry/app/api/handler/cargo/*_test.go in package cargo_test). No consistent project-wide policy.
  • Helper packages:
    • app/testing/ — present as a directory with testing.go and integration/integration.go but both files contain only the package declaration; effectively empty stubs for future use.
    • registry/app/api/controller/mocks/ — mockery-generated mocks for the registry controller interfaces (e.g., Controller, ArtifactRepository, Authorizer, ~30 files). Generated with mockery v2.53.3, each file carries // Code generated by mockery.
  • Fixtures: registry/app/api/controller/metadata/testdata/ — JSON fixture files for registry metadata controller tests. No other testdata directories found in the main application.

Test patterns#

Table-driven tests#

  • Prevalence: Heavy — 577 occurrences across the codebase. Dominant pattern wherever tests exist.
  • Style: Anonymous struct slice with a name field, iterated with t.Run(test.name, ...).
  • Example: store/database/dbtx/runner_test.go:31 — tests := []struct{ name string; fn func(...); errCommit error; ... }{} with 10 cases covering commit, rollback, panic, and context-cancellation scenarios in TestWithTx.
  • Variant: Some test functions use subtests without a slice — just sequential t.Run(...) blocks for labeling (e.g., audit/context_test.go — t.Run("returns IP when present", ...) directly). This avoids struct overhead for simple property-based checks.

Mocking approach#

  • Two distinct strategies depending on subsystem:

    1. Mockery-generated mocks (registry package only):

    • Located in registry/app/api/controller/mocks/.
    • Generated from testify/mock-based structs. Each mock embeds mock.Mock and implements the interface with _m.Called(...) delegation.
    • Test files import these mocks and use On(...).Return(...) + AssertExpectations(t).
    • Example: registry/app/api/handler/cargo/upload_test.go:67 — mockCtrl.On("UploadPackage", ctx, info, mock.AnythingOfType(...), mock.Anything).Return(resp, nil).

    2. Manual functional mocks (core application):

    • Hand-written structs with function-field callbacks: struct { FindByRefFn func(ctx context.Context, ref string) (*types.SpaceCore, error) }.
    • Seen in app/services/usage/middleware_int_test.go:168 — SpaceFinderMock and MetricsMock use this pattern. The struct method delegates to the function field.
    • Also seen in store/database/dbtx/runner_test.go:168 — dbMock and txMock are hand-written interface implementations that track call state (.committed, .rollback booleans) without any mock framework.

    3. Interface satisfaction assertion:

    • var _ transactor = (*dbMock)(nil) compile-time interface check (runner_test.go:176). Common in the codebase for verifying mock correctness at compile time.

Integration tests#

  • Present: Yes, but extremely limited — only one file found: app/services/usage/middleware_int_test.go.
  • Separation: //go:build integration build tag (plus the older // +build integration dual annotation for pre-1.17 compatibility).
  • How: The integration test (TestUploadDownloadMiddleware) spins up a real chi HTTP server in a goroutine, then fires concurrent HTTP requests against localhost:8080. It uses a waitServer() helper with retry logic to wait for the server to be ready.
  • External dependencies: Expects a running HTTP listener — not testcontainers, not dockertest. Database integration tests are absent; the database layer is tested with in-memory mocks (see dbMock above).
  • No *_e2e_test.go files found anywhere in the repository.

Test quality observations#

What’s done well#

  • Table-driven tests are thorough where they exist. The dbtx/runner_test.go covers 10 edge cases for transaction semantics (panic recovery, commit-after-cancel, double-finish), all within one table. The URL backfill tests in cli/operations/server/config_test.go cover 17 scenarios exhaustively.
  • Interface satisfaction at compile time. var _ Interface = (*Mock)(nil) is used systematically in test files to catch drift between the mock and its interface.
  • Build tag separation. The single integration test file cleanly separates itself from unit tests using //go:build integration, so go test ./... never requires a live server.
  • HTTP handler tests use net/http/httptest. 18 files test HTTP handlers without a live server. The cargo upload test exercises success, bad payload, controller error, and JSON encoding failure paths — a thorough handler test.
  • Dual in-memory/Redis broker makes stream tests possible without Redis. The stream/MemoryBroker is used in unit tests while production uses Redis; the common interface makes this transparent (noted in patterns analysis).
  • testify/require for fast-fail. Tests use require.NoError / require.Equal at the start to abort immediately on setup failure, avoiding cascading assertion noise.

What could improve#

  • Test coverage is sparse for a large codebase (8%). Critical subsystems — git/, app/pipeline/, app/router/, events/, store/database/ (main store) — have few or no tests. The Wire-generated DI chain (cmd/gitness/wire_gen.go) is untested by definition.
  • No CI test execution. The only GitHub Actions workflow (ci-lint.yml) runs golangci-lint — there is no go test step in CI. Tests are not automatically run on pull requests.
  • app/testing/ is a placeholder. The app/testing/testing.go and app/testing/integration/integration.go files contain only the package declaration — no shared test helpers, fixtures, or builder utilities exist for the core application. Each test package reinvents its own mocks.
  • No database integration tests with testcontainers. The database layer (SQLite/PostgreSQL) is tested with in-memory mock structs rather than against a real schema. Schema migration correctness and SQL query results are not validated in any automated test.
  • Inconsistent mock strategy. The registry sub-module uses mockery-generated mocks; the core app uses hand-written functional mocks; some tests use neither. This inconsistency increases maintenance burden.
  • Integration test requires manual setup. The one integration test hits localhost:8080 with no test harness to provision or tear down the service — it will fail silently in any environment without the right pre-conditions.

Patterns worth emulating (for the book)#

  • Table-driven tests with panic recovery (dbtx/runner_test.go:130-141): the test wraps the call in an anonymous function with defer recover() to test that panics inside transactions trigger rollback. A clean pattern for testing error paths that involve panic.
  • Compile-time interface assertions in test files (var _ Interface = (*Mock)(nil)): zero runtime cost, catches interface drift at go build time.
  • testify/mock with mock.MatchedBy (upload_test.go:99): using a predicate function to assert on error message content without exact string matching — more resilient than mock.Anything but less brittle than exact equality.
  • waitServer retry helper (middleware_int_test.go:240): the exponential-backoff server readiness poll with t.Helper() decoration is a clean pattern for any integration test that starts a background service.