Buildkite Agent — Overview#

Identity#

Purpose#

The Buildkite Agent is a cross-platform CI/CD build runner that polls the Buildkite SaaS platform for work, executes build jobs on user-owned infrastructure, and reports results (logs, exit codes, artifacts) back to Buildkite. It is designed to run on any infrastructure — bare metal, VMs, containers, or Kubernetes — giving teams full control over their build environment while delegating scheduling and orchestration to the hosted Buildkite service.

Significance#

Buildkite is a widely-adopted CI/CD platform used by companies including Shopify, PagerDuty, and Atlassian. The agent is the bridge between Buildkite’s hosted control plane and the user’s own compute, making it a critical piece of infrastructure in many production pipelines. Its Go implementation, cross-platform support (Linux, macOS, Windows on x86/arm64), and plugin extensibility model have made it a reference implementation for self-hosted CI agents.

Key metrics#

  • Go files: 369
  • Top-level directories: agent, api, clicommand, cliconfig, core, env, internal, jobapi, kubernetes, lock, logger, metrics, process, status, templates, test, tracetools, version
  • Direct dependencies: ~65 (from go.mod first require block)
  • First commit / age: 2014 (per MIT license copyright), ~12 years old

Notable characteristics#

  • Multi-cloud artifact storage: Built-in support for AWS S3, Azure Blob Storage, and Google Cloud Storage for artifact upload/download, with AWS KMS and GCP KMS for artifact signing.
  • Rich observability stack: Ships with OpenTelemetry tracing (OTLP gRPC/HTTP), DataDog APM (dd-trace-go), Prometheus metrics, and OpenTracing compatibility — four separate telemetry systems in one binary.
  • Job API via Unix socket: Exposes a local HTTP API (jobapi + internal/agentapi + internal/socket) to running jobs, allowing build steps to interact with the agent process itself (retrieve OIDC tokens, set metadata, upload artifacts) without going through the network.
  • Plugin system: The agent/plugin sub-package implements a checkout-based plugin mechanism where plugins are Git repositories cloned at job time, with hook scripts executed at defined lifecycle points.
  • Kubernetes-native mode: The kubernetes package implements a distinct execution model where the agent orchestrates containers in a Kubernetes pod rather than running subprocesses, demonstrating a pluggable execution backend design.