fzf — Testing#

Test metrics#

  • Test files: 23
  • Source files (non-test .go): 57
  • Ratio (test files / source files): ~0.40
  • Test frameworks: stdlib testing only — no testify, gomock, ginkgo, or any third-party test library

Test organization#

  • Placement: Same package (package fzf, package algo, package util) throughout. No _test (black-box) packages — all tests have access to unexported identifiers.
  • Helper packages: None — no testutil/, mock/, or fake/ directories. Test helpers are local functions within each _test.go file (e.g., assertMatch/assertMatch2 in algo_test.go, assert in merger_test.go).
  • Fixtures: No testdata/ directories. Test inputs are inline literals and programmatically generated data (random results, constructed byte slices).

Test patterns#

Table-driven tests#

  • Prevalence: Moderate — used in ~5 of the 23 test files, always where the input space is enumerable and parallel-case structure is natural.
  • Style: Both anonymous struct slices ([]struct{ name, s string; b1, b2 byte; want int }) and maps (map[string]string). Named fields with name for subtests and want for expected value — standard Go style.
  • Example: src/algo/indexbyte2_test.go:8tests := []struct { name string; s string; b1, b2 byte; want int }{ ... } with t.Run(tt.name, ...) for the SIMD path. src/options_test.go:519testCases := []struct{...} for option parsing.

Mocking approach#

  • Strategy: None. No mock library is used anywhere. Dependencies are tested indirectly — e.g., pattern_test.go and merger_test.go construct real Item and Result objects rather than stubs.
  • Implication: The tight package coupling (all tests are white-box) and the absence of interface-heavy design in core internals means mocking is rarely necessary. The components are small and deterministic enough to test directly.

Integration tests#

  • Present: Yes — but implemented in Ruby, not Go.
  • How: test/runner.rb drives a full fzf binary via a tmux session. Test files cover core UI (test_core.rb), preview (test_preview.rb), server (test_server.rb), shell integration (test_shell_integration.rb), layout (test_layout.rb), and raw mode (test_raw.rb). The tmux approach allows testing terminal I/O behavior that cannot be exercised from unit tests.
  • Separation: Entirely separated from the Go test tree — lives in test/ with a Ruby runtime dependency. Invoked via make install && ruby test/runner.rb in CI.

Fuzz tests (notable)#

  • Present: Yes — FuzzIndexByteTwo and FuzzLastIndexByteTwo in src/algo/indexbyte2_test.go:145–168.
  • Corpus seeds: Three seed inputs each (empty, single-byte, same-byte cases).
  • Oracle: Each fuzz test compares the SIMD/asm implementation against a pure-Go loop reference (loopIndexByteTwo, refLastIndexByteTwo). This is a differential fuzzing pattern — the reference is intentionally slower but obviously correct.
  • CI integration: Both fuzz targets run in CI with -fuzztime=5s on every push (linux.yml:48–51). Not just regression-seeded — actual fuzzing runs on CI.

Benchmarks#

  • Present: Yes — 9 benchmark functions across algo/indexbyte2_test.go (6) and ansi_test.go (3).
  • Style: Parameterized via shared bench* helpers that run multiple implementations (asm vs stdlib vs loop) as sub-benchmarks, enabling direct performance comparison in a single go test -bench run.
  • Example: BenchmarkIndexByteTwo_10/100/1000 — each calls benchIndexByteTwo(b, size, pos) which runs IndexByteTwo (asm), refIndexByteTwo (2×IndexByte), and loopIndexByteTwo (naive loop) as sub-benchmarks at three input sizes.

Exhaustive / property-style tests#

  • Example: indexbyte2_test.go:40–73 — a nested loop tests IndexByteTwo for every buffer length from 0 to 256, inserting a match at every position, and comparing against the reference. This is manual exhaustive verification that would normally be done by a property-based test framework. Combined with fuzz tests, the SIMD implementation is exceptionally well validated.

Oracle / reference-implementation pattern#

  • Example: ansi_test.go:25–66testParserReference compares the hand-written nextAnsiEscapeSequence() parser against a reference regex (ansiRegexReference) character by character on each test string. The hand-written parser is faster but harder to reason about; the regex is the ground truth. TestNextAnsiEscapeSequence runs both and diffs the output.

Concurrency-aware tests#

  • Example: util/eventbox_test.goTestEventBox spawns a goroutine that fires events in three distinct phases, synchronizing with the test goroutine via a raw chan bool handshake. Verifies both coalescing behavior (three rapid EvtSearchNew events collapse to one) and the total accumulated value. The test is self-contained and deterministic via explicit synchronization.

Test quality observations#

What’s done well#

  • Zero external test dependencies. The entire unit test suite runs with go test ./... and no installation step. No testify, no gomock — dependency count stays at zero.
  • Differential / oracle testing for performance-critical code. The SIMD IndexByteTwo implementation is tested against a plain-Go loop reference, making it impossible for the optimized path to diverge silently. This is more rigorous than testing against hardcoded expected values.
  • Fuzz tests in CI. Running fuzz tests for 5 seconds per target on every PR is an unusually disciplined practice. Most projects commit fuzz corpora and run them only in regression mode; fzf actually fuzzes on CI.
  • Exhaustive boundary coverage for SIMD code. The 0–256 size sweep in indexbyte2_test.go specifically targets SIMD block boundaries (16, 32, 64 bytes), which is exactly where off-by-one errors in SIMD code tend to appear.
  • Realistic integration tests via tmux. The Ruby test suite exercises fzf as a user would — typing keystrokes, reading terminal output — which no unit test can replicate. This is the right approach for a terminal UI tool.

What could improve#

  • No coverage for the coordinator / event loop. core.go (the Run() function and the main event dispatch loop) has no unit test. Testing this would require either integration tests or a significant redesign. The Ruby test suite covers the behavior, but not the internal state.
  • No property-based testing library. The exhaustive loops in indexbyte2_test.go are manually written. A library like gopter or rapid would express the same intent more concisely and generate more interesting inputs automatically (though fuzz tests partially compensate for this).
  • No test for the concurrency model under stress. The EventBox test is single-threaded in the producer. A concurrent stress test (multiple producers racing) would give more confidence in the Mutex/Cond implementation.
  • Terminal and Reader components are untested at the unit level. terminal.go (the largest file at ~5700 lines) has only a terminal_test.go that tests replacePlaceholder — a pure string-manipulation function. The rendering, key-binding dispatch, and preview logic are covered only by integration tests.

Patterns worth emulating#

  • Oracle testing for optimized implementations: Write the obvious-but-slow version first, keep it in the test file as a reference function, and use it to verify the fast version. indexbyte2_test.go is the canonical example.
  • Fuzz tests with -fuzztime in CI: A short fuzz run (5s) catches regressions and occasionally finds new bugs without requiring a dedicated fuzzing infrastructure. The pattern is: fuzz corpus seeds → CI fuzz run → check in any new crashes as regression tests.
  • Benchmark sub-tables: benchIndexByteTwo runs three competing implementations in a single benchmark function, producing output that directly shows the speedup. This is the right way to benchmark algorithmic alternatives — the comparison is built into the benchmark itself.