restic — Testing#

Test metrics#

  • Test files: 211
  • Source files (non-test): 325
  • Ratio (test / source): ~0.65 — notably high for a systems-level tool
  • Test frameworks: stdlib testing only — no testify, gomock, ginkgo, or any third-party assertion library

Test organization#

Placement#

Mixed strategy, deliberately chosen per test type:

  • _test package (black-box): most backend tests, filter tests, integration tests in cmd/restic
  • Same package (white-box): internal tests that need access to unexported fields, e.g., local_internal_test.go (package local) alongside local_test.go (package local_test) in the same directory. Both styles coexist in the same package directory when needed.

Helper packages#

  • internal/test — project-wide test utility package. Contains:

    • Assert, OK, OKs — simple condition/error assertion helpers styled like testify but handrolled
    • Equals[T any] — generic equality check using reflect.DeepEqual
    • Random(seed, count int) []byte — deterministic pseudo-random data generator (seeded, reproducible)
    • TempDir(t) — creates a temp directory registered with t.Cleanup
    • SetupTarTestFixture — extracts a .tar.gz or .bzip2 archive fixture into a temp dir
    • Env(t, repoFixture) — creates a full test repo environment from a tar fixture; returns path + cleanup closure
    • Chdir(t, dest) — changes working directory, returns a restore func
    • RemoveAll, ResetReadOnly — Windows-safe recursive deletion helpers
    • vars.go — all test knobs are env-var-controlled: RESTIC_TEST_PASSWORD, RESTIC_TEST_CLEANUP, RESTIC_TEST_TMPDIR, RESTIC_TEST_INTEGRATION, RESTIC_TEST_FUSE, RESTIC_TEST_S3_SERVER, RESTIC_TEST_REST_SERVER, RESTIC_TEST_DISALLOW_SKIP

  • internal/backend/mock — hand-rolled mock for backend.Backend. Uses function-field structs: every method is a FnField func(...) that defaults to a no-op or errors.New("not implemented") if nil. Callers override only the methods they need: be.SaveFn = func(...) error { ... }. A compile-time assertion var _ backend.Backend = &Backend{} guards correctness.

  • internal/backend/test — a generic backend acceptance test suite. Suite[C any] is a generic struct parameterized on the backend config type. It discovers and runs all methods named Test* / Benchmark* via reflection (reflect.TypeOf(s).NumMethod()). Every official backend (local, sftp, s3, rest, rclone, cache, sema) calls suite.RunTests(t) in a single line, getting full CRUD/concurrency/edge-case coverage for free.

  • internal/repository/testing.go — production-package test helpers (not a _test file, so they can be imported by other packages):

    • TestRepository(t) — returns a fully initialised in-memory repository, fast and hermetic
    • TestRepositoryWithBackend(t, be, version, opts) — parameterised version
    • TestUseLowSecurityKDFParameters(t) — replaces scrypt params with N=128, R=1, P=1 using sync.Once, making KDF cost negligible without per-test boilerplate
    • TestAllVersions(t, fn) / BenchmarkAllVersions(b, fn) — parametrize over all supported repo format versions
    • TestFromFixture(t, repoFixture) — opens a repo from a tar archive fixture
    • TestCheckRepo(t, repo) — runs the full integrity checker on a test repo

  • internal/backend/retry/testing.go — TestFastRetries() sets global retry delays to 1 ms, preventing slow exponential backoffs in tests.

  • internal/restic/config.go — TestDisableCheckPolynomial(t) disables the CDC polynomial validation, which is expensive and unnecessary for most tests.

Fixtures#

  • cmd/restic/testdata/ — contains backup-data.tar.gz: a real directory tree used as backup source in integration tests. testSetupBackupData extracts it at test time.
  • internal/backend/testdata/, internal/checker/testdata/, internal/filter/testdata/, internal/repository/testdata/ — per-package fixture repositories and data files.
  • Fixtures are tar archives extracted at test start; no Docker or external daemon required for the majority of tests.

Test patterns#

Table-driven tests#

  • Prevalence: Heavy. 228 usages of t.Run/tt.Run/testCases patterns in test files.
  • Style: Anonymous struct slices are the dominant form:
    var matchTests = []struct {
    pattern string
    path    string
    match   bool
}{
    {"*.go", "/foo/bar/test.go", true},
    {"*.c", "/foo/bar/test.go", false},
    ...
}
    (internal/filter/filter_test.go has 100+ rows in a single table.)
  • Subtests: t.Run(name, fn) used consistently when a name is available; some older tests iterate without subtests.
  • Example: internal/filter/filter_test.go:14 — large matchTests slice; internal/restic/parallel_test.go — concurrent operation tables.

Mocking approach#

  • Strategy: Hand-rolled function-field mocks, not generated. No gomock, no mockery.
  • Pattern: The mock.Backend struct has one exported FnField per interface method. If nil, the method either no-ops or returns errors.New("not implemented"). Tests set only the fields they care about. This is lighter-weight than gomock and avoids code generation in the build.
  • Example: internal/backend/mock/backend.go — SaveFn func(...), ListFn func(...), etc. Used heavily in backend decorator tests (retry, sema, cache layers).

Integration tests#

  • Present: Yes, an extensive suite.
  • Separation: By naming convention — cmd_*_integration_test.go in cmd/restic/. There are 20 integration test files, one per major command (backup, restore, check, forget, prune, copy, diff, rewrite, find, ls, list, tag, key, mount, snapshots, generate, init, recover, repair-index, repair-snapshots).
  • How: All-in-process — no Docker, no external services for core integration tests. The withTestEnvironment helper creates a temporary directory tree with a live local-backend repository. It applies three critical speed knobs: TestUseLowSecurityKDFParameters, TestDisableCheckPolynomial, TestFastRetries. Tests call the same runBackup, runRestore, runCheck, etc. functions as the real CLI, with a captured termstatus output.
  • External backend tests (optional): S3 (Minio), REST server, SFTP — enabled by env vars (RESTIC_TEST_S3_SERVER, RESTIC_TEST_REST_SERVER). The CI workflow (tests.yml) starts Minio and a rest-server on Linux runners to exercise these paths. A RESTIC_TEST_DISALLOW_SKIP env var can force specific tests to fail rather than skip, ensuring CI doesn’t silently drop coverage.
  • Race detector: The CI matrix includes a dedicated Linux (race) job running go test -race. The errgroup + channel concurrency patterns are specifically written to be race-free.
  • Backdoor hook: global.Options.BackendTestHook func(backend.Backend) (backend.Backend, error) — allows tests to wrap the backend with inspection or fault injection without changing production code. Used in integration tests to enforce that each file type is listed at most once (newOrderedListOnceBackend).

Backend acceptance suite pattern#

  • internal/backend/test.Suite[C] uses reflection to discover Test* methods on the suite struct. Each backend calls suite.RunTests(t) once; the suite creates a fresh backend, runs 15+ behavioral tests as subtests, then cleans up. This is the closest analogue to a parameterized test suite available without third-party frameworks.
  • Multi-version parametrization: TestAllVersions(t, fn) iterates over MinRepoVersion..MaxRepoVersion, calling the test function for each. This ensures every supported repository format is exercised without duplicating test code.

Test quality observations#

What’s done well#

  • Zero third-party test dependencies: The entire test suite uses only stdlib testing. The custom internal/test helpers are purpose-built and minimal. This keeps go test ./... fast and dependency-free.
  • In-process integration tests: Running real command implementations against a real (local-disk) repository gives high confidence without Docker overhead. Tests cover round-trips: backup → restore → diff → check.
  • Speed knobs without test-only binaries: TestUseLowSecurityKDFParameters, TestFastRetries, TestDisableCheckPolynomial are production-package functions (not in _test.go) that can be called by any package’s tests. This avoids copying KDF setup boilerplate across 20 integration tests.
  • Generic backend suite: Suite[C any] with reflection-based discovery is a clean solution to the “run the same test against N implementations” problem without inheritance or code generation.
  • Environment-variable configurability: All external test dependencies (S3, REST, SFTP) can be injected via env vars. Tests skip gracefully when the service is absent, but CI can enforce full runs with RESTIC_TEST_DISALLOW_SKIP.
  • Race detector in CI: Dedicated -race matrix job gives confidence that the errgroup/channel concurrency patterns are correct.
  • Fixture archives: Using .tar.gz fixtures for both test repositories and backup source data provides stable, reproducible test inputs.

What could improve#

  • No test names in some tables: Several older table-driven tests iterate without subtests, making failure messages like FAIL: index 23 hard to debug.
  • Reflection-based suite discovery: The Suite.testFuncs reflection loop is fragile — method signature mismatches produce only a log warning, not a compile error. A go generate-based approach or explicit registration would be more robust.
  • Test helpers in production packages: Files like internal/repository/testing.go and internal/backend/retry/testing.go ship in the non-test binary (testing package imported in non-_test.go files). This is deliberate but adds a testing import to the production binary. Restic accepts this trade-off for usability.
  • No fuzz tests: The CDC chunker, crypto layer, and backend parsers would benefit from go test -fuzz harnesses; none are present.

Patterns worth emulating#

  • The internal/test package approach: a small set of handrolled assertion helpers (Assert, OK, Equals[T]) is all that’s needed and avoids testify as a dependency.
  • The function-field mock pattern (mock.Backend) for mocking an interface: zero codegen, easy to extend, and the nil-check default behavior reduces test setup boilerplate dramatically.
  • The test knob in production code pattern: TestUseLowSecurityKDFParameters lives in internal/repository (not _test.go) so it can be called from anywhere. For libraries with expensive-but-adjustable parameters (crypto, networking), this is a powerful technique.
  • The generic acceptance suite (Suite[C any]): demonstrates that a single well-designed test suite can verify all implementations of an interface without per-implementation duplication, using Go generics + reflection.
  • The BackendTestHook in global.Options: allowing tests to wrap the backend with a hook function at the composition root is a clean fault-injection mechanism that doesn’t pollute production types with test conditionals.
Backward restic — API Surface Syncthing — Overview Forward