Buildkite Agent — Structure#

Layout pattern#

Custom (domain-layered, no cmd/ subdirectory)

The project produces a single binary directly from a root-level main.go rather than using the conventional cmd/<binary>/main.go pattern. All CLI command wiring lives in clicommand/, deep business logic lives in agent/ and internal/, and a small number of public-facing library packages sit at the top level. The layout reflects the project’s age (2014) and its single-binary nature — there was no need for a cmd/ layer when there is only one binary.

Directory map#

buildkite-agent/
├── main.go                  # Single binary entry point; delegates to clicommand
├── agent/                   # Core agent runtime: pool, worker, job runner, log streaming
│   ├── integration/         # Integration tests for agent-level behaviour
│   └── plugin/              # Plugin definition and resolution logic
├── api/                     # REST client for the Buildkite SaaS API (typed structs + HTTP)
│   └── proto/               # Protobuf definitions + generated code for Agent API
├── clicommand/              # urfave/cli command wrappers (one file per command/subcommand)
├── cliconfig/               # Config file parsing (INI-style .cfg file loader)
├── core/                    # Unstable public library API (agent client, controller abstractions)
├── env/                     # Environment variable collection and manipulation helpers
├── internal/                # All private implementation packages
│   ├── agentapi/            # Server-side of the local agent HTTP API (OIDC, metadata, etc.)
│   ├── agenthttp/           # HTTP middleware utilities for the local API
│   ├── artifact/            # Artifact upload/download across S3, GCS, Azure, Buildkite CDN
│   ├── awslib/              # AWS SDK helpers (S3, KMS, IAM)
│   ├── bkgql/               # GraphQL client for Buildkite API (signing/verification queries)
│   ├── cache/               # Build cache save/restore logic
│   ├── cryptosigner/        # Abstract signing interface + AWS KMS and GCP KMS adapters
│   ├── e2e/                 # End-to-end test harness and fixtures
│   ├── experiments/         # Feature-flag / opt-in experiments system
│   ├── file/                # File utility helpers
│   ├── job/                 # Job execution: checkout, command, hook scripts, env capture
│   │   ├── hook/            # Hook script lifecycle (binary detection, env diff, wrapping)
│   │   ├── githttptest/     # Fake git HTTP server for integration tests
│   │   └── integration/     # Job execution integration tests
│   ├── mime/                # MIME type detection (generated lookup table)
│   ├── olfactor/            # Secret/value detection (olfaction = smell = sniff for secrets)
│   ├── osutil/              # Cross-platform OS utilities (file permissions, signals)
│   ├── ptr/                 # Generic pointer helpers
│   ├── race/                # Race detector utilities
│   ├── redact/              # Log redaction engine (streaming replacement of secrets)
│   ├── replacer/            # Byte-stream multi-pattern replacer (Aho-Corasick-like)
│   ├── secrets/             # Secret resolution (Buildkite Secrets API)
│   ├── self/                # Self-update helper
│   ├── shell/               # Shell abstraction: command execution, env, PTY management
│   ├── shellscript/         # Shell script generation helpers
│   ├── socket/              # Unix domain socket server/client for local Job API
│   ├── stdin/               # Stdin detection utilities
│   ├── system/              # Sysinfo (OS/arch detection)
│   ├── tempfile/            # Temp file creation helpers
│   └── trie/                # Trie data structure for prefix matching
├── jobapi/                  # Public Job API client (used by build steps to call back to agent)
├── kubernetes/              # Kubernetes-native execution backend (pod orchestration)
├── lock/                    # Distributed lock primitives (via local socket)
├── logger/                  # Logging interfaces and adapters (text, JSON, OpenTelemetry)
├── metrics/                 # Prometheus metrics registration and collection
├── process/                 # Subprocess execution with PTY, streaming, signal handling
├── status/                  # Agent status page (expvar-style introspection endpoint)
├── templates/               # launchd plist templates for macOS service installation
├── tracetools/              # OpenTelemetry/OpenTracing span propagation utilities
├── version/                 # Version string management
├── packaging/               # Non-Go packaging artefacts
│   ├── docker/              # Dockerfiles (alpine, alpine-k8s, sidecar, ubuntu-*)
│   └── linux/               # Debian/RPM packaging scripts and configs
├── scripts/                 # Shell scripts for building, packaging, publishing releases
└── test/                    # Shared test fixtures (hook scripts, etc.)

Entry points#

The project produces exactly one user-facing binary. The two other main.go files are test helpers, not production binaries.

Package organization#

• Internal packages (internal/):

  • internal/job — job execution pipeline (checkout, commands, hooks, env capture)
  • internal/job/hook — hook script lifecycle management
  • internal/artifact — multi-backend artifact upload/download (S3, GCS, Azure, Buildkite CDN)
  • internal/shell — cross-platform subprocess and PTY management
  • internal/socket — Unix domain socket server/client for the local Job API
  • internal/agentapi — HTTP handler for the in-process agent API
  • internal/redact + internal/replacer — streaming log redaction engine
  • internal/cryptosigner — signing abstraction with AWS KMS and GCP KMS implementations
  • internal/experiments — feature flag system
  • internal/e2e — end-to-end test infrastructure
  • Misc small utilities: env, file, mime, osutil, ptr, race, secrets, self, shellscript, stdin, system, tempfile, trie

• Public packages (top-level, no pkg/ directory):

  • agent/ — agent runtime (pool, worker, job runner); heavily used internally, not designed for external import
  • api/ — typed HTTP client for the Buildkite REST API; usable externally but primarily internal
  • core/ — explicitly unstable library API (doc says “not stable, use at own risk”)
  • jobapi/ — Job API client intended for use by build steps (public-facing library)
  • clicommand/ — CLI command wiring; internal in practice
  • cliconfig/ — config file loader; internal in practice
  • env/, logger/, metrics/, lock/, process/, tracetools/, version/ — domain utilities

• Layering: The project follows a loose layered approach: main.go → clicommand → agent → internal/*. The agent package is the orchestration layer; internal/job and internal/shell are the execution layer; api is the network layer. There is no strict clean-architecture enforcement — agent imports both internal and api directly.

Build system#

• Build tool: Bazel (primary), plain go build (also supported), shell scripts in scripts/
• Key targets:
  • bazel build //:buildkite-agent — produces the static binary (pure Go, CGO disabled via pure = "on")
  • scripts/build-binary.sh — wraps go build for CI/release
  • scripts/build-debian-package.sh, scripts/build-rpm-package.sh — Linux packaging
  • scripts/build-github-release.sh — GitHub release artefact assembly
• Docker: Yes, multi-stage implied. Six image variants: alpine, alpine-k8s, sidecar, ubuntu-20.04, ubuntu-22.04, ubuntu-24.04. Each Dockerfile copies a pre-built binary from a base image and adds config.
• Bazel workspace: Uses gazelle for BUILD.bazel generation. MODULE.bazel + MODULE.bazel.lock indicate Bzlmod (Bazel 6+ module system).
• go:generate: Used in main.go to regenerate the MIME type lookup table (internal/mime/generate.go).

Notable structural decisions#

1. Root-level main.go instead of cmd/: With a single binary, the project skips the cmd/ directory entirely. main.go is a thin 78-line file that delegates everything to clicommand. This keeps the project root clean but deviates from the Standard Go Layout convention.

2. clicommand/ as a separate package from agent/: Each CLI subcommand gets its own file in clicommand/ (e.g., agent_start.go, artifact_upload.go). This creates a clean separation between CLI argument parsing and business logic — the command files are purely wiring.

3. internal/ for all private implementation, no pkg/: The project never adopted the pkg/ convention. Everything meant for external consumers lives at the top level; everything internal goes in internal/. This is the more modern Go convention.

4. core/ as an explicitly unstable public library: An unusual and honest design choice: the package doc explicitly warns that the API is not stable. This allows the team to export some types for external use (e.g., embedding the agent) while reserving the right to break them.

5. Dual execution models side by side: The agent/ package handles standard subprocess-based execution; the kubernetes/ package implements a completely different pod-orchestration execution model. Both are registered as CLI commands (BootstrapCommand vs KubernetesBootstrapCommand) at the same level, demonstrating a pluggable execution backend pattern without a formal plugin interface.

6. Bazel alongside go build: The presence of BUILD.bazel files throughout and MODULE.bazel at the root shows adoption of Bazel for reproducible builds, while keeping standard go build support. This is unusual for a project of this size but reflects Buildkite’s internal tooling preferences.