Skip to content

How to design GitHub Actions

The first sections are free. The deep dives, level expectations and quiz are part of Pro.

On this page
Asked at
Practice this question

The problem

GitHub Actions turns a push into workflow runs, splits each run into jobs, and hands those jobs to a shared pool of machines while developers watch the logs. The answer turns on keeping three things apart because they scale differently: taking events in durably, deciding which job runs next when thousands of tenants share one pool, and moving logs, artifacts and caches around the scheduler rather than through it. GitHub's published figures anchor the arithmetic: 71 million jobs each day, and from its own internal fleet an average of about 8.3 minutes per job, which is the one job-length figure this guide carries through its estimates.

Clarifying questions

AskWhy it mattersA typical answer

How do we learn about a push: does the source host call us, or do we poll it?

A webhook decides the shape of intake: a receiver that must acknowledge fast, deal with a repeated delivery, and survive a host that resends.

The source host sends a webhook per event with a delivery id. It may send the same delivery again, so one push must start one run.

What scale: repositories, pushes a second, jobs a day, and how long a job runs?

Pushes a second sizes the intake; jobs a day and job length together size the runner pool, which is the expensive part.

About 10 million repositories, about 10 pushes a second with bursts, and GitHub's own figure of 71 million jobs each day. Jobs on hosted runners can run up to 6 hours; no average is given, so estimate one and say where it comes from.

Are the machines ours, the customer's, or both?

Customer-owned runners sit behind firewalls and change how jobs reach a machine; hosted runners make isolation and warm capacity our problem.

Both. Hosted runners by default, and customers may register their own machines, matched to jobs by labels.

Do public repositories run workflows for pull requests from strangers?

If yes, the runner executes code written by anyone, and isolation and secret handling move from nice-to-have to a core requirement.

Yes. Assume a public repository accepts pull requests from anyone and runs their workflow, perhaps after a maintainer approves the first run from a new contributor.

How uneven are tenants? What does the worst burst look like?

Fairness is the center of this question. One tenant's burst size tells you whether a single queue is acceptable.

One organization can queue 10,000 jobs at merge time while a small team queues 5. The small team should not wait behind the big one.

Can jobs have side effects, such as deploying, and may the system rerun them on its own?

A retried build is harmless; a retried deploy may not be. The answer decides whether a lost runner's job goes straight back in the queue.

Yes, jobs deploy. Retries are expected after failures, so the design has to make a repeated deploy step safe or stop it.

The requirements menu

RequirementScopeWhy

A push, pull request, tag or schedule starts the workflows defined in YAML files in the repository

Core

The entry point. It forces a durable intake and a service that reads the YAML at the triggering commit, since the file can differ between branches.

Each run splits into jobs with dependencies; each job runs on a matching runner, hosted or the customer’s own

Core

The job graph is what the scheduler works on, and labels are how a job finds a machine that can run it.

Developers watch live logs, and download logs and artifacts after the run

Core

The read path developers see most. It forces a streaming path for live lines and a storage path for finished output, and they shouldn't be the same box.

Retry, cancel and time out runs and jobs, and report status back to the commit or pull request

Core

Cancel and retry are state transitions on the job record, so job state has to live somewhere the scheduler can reach, not on the runner.

Cache dependencies and build outputs between runs

Core

A build often redoes work another run already did, such as fetching dependencies or compiling unchanged code. Caching is where the speed comes from, and where a wrong hit comes from too.

Matrix jobs that expand one job definition into many

Optional

Cheap to add on the workflow service, since it just writes more job rows. Worth a sentence because it shows the job graph is data, not code.

Concurrency groups that cancel a superseded run on the same branch

Optional

Can save runner time where many pushes land on the same branch in quick succession. Take it if the interviewer presses on cost; otherwise name it and move on.

Approval gates on deployment environments

Optional

A paused job waiting for a human is a state, not a running process. Take it when the deploy discussion comes up, because it ties into who can release what.

A test-sharding mode that splits a huge test suite across many workers

Optional

This is the Google variant of the question. Leave it unless asked; it changes the unit of work from a job to a shard.

The build tools and test frameworks themselves

Out of scope

The runner executes whatever the step says. What happens inside the step is the user's business.

Code review and the source host

Out of scope

We receive events from it and post status back to it; everything else about it is another system.

A marketplace of shared actions

Out of scope

A catalog and a supply-chain problem of its own. Mention that a pinned action is just more code the job runs, then park it.

Billing for minutes

Out of scope

Metering job minutes is a separate pipeline fed from the job table's start and end times. Say where the data comes from and stop.

Non-functional requirements

RequirementWhat it forces

Tens to hundreds of thousands of runs at once across many tenants

The runner pool, not the control plane, is the big thing to size: one job per machine means hundreds of thousands of machines or VMs in flight. Job state has to be in a store that many scheduler instances can read, and the pool has to grow and shrink with demand.

Fair scheduling: one organization queuing 10,000 jobs does not starve one queuing 5

Ready jobs can't sit in one global line. Queues are kept per tenant, each tenant has a cap on jobs running at once, and idle runners are shared across the tenants that are waiting.

Strong isolation: one workflow cannot read another tenant’s secrets or files

A fresh machine or VM per job that is destroyed afterward, secrets issued per job and scoped to one repository and one environment, and no secrets at all for runs triggered from a fork.

A typical pull request reaches a deploy decision in tens of minutes

Queue wait and machine start both count against this budget, so the design keeps warm capacity ready and caches dependencies and build outputs instead of rebuilding them.

A push event is never lost, and never starts the same run twice

The receiver writes the event durably, keyed by the host's delivery id with a unique constraint, before it acknowledges, and the workflow service picks events up from that record rather than from the request.

Estimates

QuantityWorkingResultWhat it sizes

Average job length, from a figure GitHub states

GitHub's own CI on larger runners at peak: 125,000 build minutes an hour across about 15,000 jobs an hour (GitHub blog, 2023). 125,000 ÷ 15,000 ≈ 8.3 minutes per job

About 8.3 minutes per job, for GitHub's internal mix in 2023

The one job-length figure the later rows use. It describes GitHub's own repositories, so treat it as an informed assumption for the whole platform and say so.

How busy GitHub's peak fleet was

4,500 concurrent runners × 60 minutes = 270,000 runner-minutes an hour available; 125,000 build minutes used; 125,000 ÷ 270,000 ≈ 46%

Under half of the fleet's minutes were used in the busiest hour

Headroom: a pool sized to its average load has no room for a burst, and a fleet that looks big is often waiting. Don't assume all runners are busy when you derive job length.

Jobs in flight at GitHub's scale

820 jobs per second (the daily average) × 8.3 minutes × 60 seconds ≈ 410,000 jobs running at once, if the platform mix resembles GitHub's own. GitHub gives no peak; an assumed peak of 3 × average gives about 1.2 million. Don't divide these by the interviewer's 10 pushes a second: that figure comes from a different scenario

About 410,000 runners busy on average, around 1.2 million at an assumed peak of three times average

The runner pool, one job per machine: the largest and most expensive part of the design.

Draining one tenant's 10,000-job burst

10,000 jobs × 8.3 minutes ≈ 83,000 runner-minutes. Under a cap of 1,000 jobs running at once (the ceiling GitHub lists as usual for one Linux larger-runner type), that is 10 waves × 8.3 minutes ≈ 83 minutes. In a single shared line with a pool of that size, a five-job team queued behind it starts after about 9 waves, roughly 75 minutes; with fair sharing between tenants it starts in the first wave

About 83 minutes for the burst under a 1,000-job cap; the five-job team waits about 75 minutes in one shared line, or about 8 minutes with fair sharing

Why per-tenant caps and fair sharing are the fix, and what the small team gains from them.

Lease and heartbeat timing

GitHub re-queues a job whose assigned runner hasn't started it inside 60 seconds. Using the same order for a running job: heartbeats at 10-second intervals, a lease that lapses after 60 seconds, so a dead runner is noticed in about a minute. A 6-hour job restarted from scratch repeats up to 6 hours of work, so a lapse must be rare and a false alarm expensive

A design choice, not a sourced figure: heartbeats at 10-second intervals and a 60-second lease; a wrong requeue can cost up to 6 hours

How long a silent runner's job waits before another machine takes it, and why deploy steps need a fencing token rather than a shorter lease.

Log volume a day

Assume 1 MB of log per job (an assumption; no source gives it). 71 million jobs × 1 MB ≈ 71 TB a day; ÷ 86,400 seconds ≈ 820 MB a second of appends on average. At an assumed retention of 90 days, 71 TB × 90 ≈ 6.4 PB. If each line were a 100-byte row, 1 MB is 10,000 rows a job, about 8.2 million row inserts a second

About 71 TB a day, 820 MB a second, roughly 6.4 PB kept at 90 days of retention

Why log lines belong in object storage, fed by a log service, and not in the main database as rows.

Cache storage ceiling

GitHub's default cap is 10 GB per repository. 10 GB × 10 million repositories = 100 PB if all 10 million repositories filled the default cap; the realized total depends on how much of the cap repositories actually use, which no source gives

About 100 PB if all repositories sat at the default cap; a repository can raise its cap, so the true ceiling is higher

Object storage with per-repository caps and eviction of entries nobody has used recently, not a database.

What a prebuilt machine image saves

GitHub reports its workflows took about 50 minutes without custom images and about 12 minutes with them: 50 − 12 = 38 minutes saved, 38 ÷ 50 ≈ 76%

About 38 minutes, roughly three quarters of the workflow

The value of warm images that already contain the source tree and compiled outputs, against the cost of rebuilding them several times a day.

Core entities

EntityWhat it holds

Event delivery

One row per webhook delivery from the source host, written before the host hears back.

Workflow

A YAML file in the repository, read at the triggering commit, that says when to run and what jobs to create.

Run

One execution of a workflow for one event: the unit developers see as a check on a commit.

Job

A node in the run's dependency graph; the thing a runner leases and executes.

Runner

A machine or VM that polls for work and runs one job at a time.

Log chunk

A slice of one step's output, addressed by its byte offset so a viewer can resume.

Blob entry

An artifact or a cache entry: metadata in the index, bytes in object storage.

The API

EndpointNotes

Inbound webhook: POST /events { repo, type, commit, delivery_id } from the source host

Verify the signature, insert the delivery with its id under a unique constraint, and answer with HTTP 202 once the row is committed. A duplicate id gets the same 202 and nothing else happens. The handler doesn't read the YAML or start jobs; that work belongs to the workflow service.

Runner: POST /runners/poll { labels } → a job assignment or nothing

A long poll opened by the runner, so a customer's machine behind a firewall needs no inbound port. The scheduler answers with a job, a lease and its expiry, a per-job token for secrets and uploads, and the job's attempt number, our own counter (not a rerun count such as GitHub's run_attempt) that in this design increases on each lease grant (a requeue after a lapsed lease as well as a rerun) so it can serve as the fencing token.

Runner: POST /jobs/{id}/logs and POST /jobs/{id}/status

Logs carry a step number and byte offset so a retried upload is harmless. Status posts also act as heartbeats and extend the lease; the server rejects them if the attempt number is stale.

GET /runs/{id} and GET /jobs/{id}/logs?offset=… for live logs

The offset lets a browser resume after a reconnect without a gap or a repeat. While the job runs, lines come from the log service; once it finishes, the same call reads from object storage.

POST /runs/{id}/cancel and POST /runs/{id}/rerun

Cancel flips the job rows and the runner learns of it on its next heartbeat response. Rerun creates a new attempt for the failed jobs, which also invalidates any lease the old attempt held.

The high-level design

webhooknotifywriteSource hostEvent receiverWorkflow serviceRuns and jobs DB

Step 1 of 4

Take the event, expand the workflow, record the run

Start with the path from a push to a set of jobs. The source host posts a webhook to an event receiver, which checks the signature and hands the event to a workflow service. That service fetches the YAML files at the triggering commit, works out which workflows the event matches, expands each into a run with its jobs and their dependencies, and writes all of that to a runs-and-jobs database. The receiver does no build work itself, so a slow workflow can't block intake. The gap: the receiver has acknowledged before anything durable exists, so a crash between the acknowledgment and the write loses the push, and a host that resends the same delivery starts a second run.

webhookappendnotifywritereadydepthleasegrowchunksuploadtokenSource hostEvent receiverEvent logWorkflow serviceRuns and jobs DBSchedulerAutoscalerRunner VM poolLog serviceObject storageSecrets service
The whole design

What breaks first

The deep dives

Queueing jobs onto runners. Pushes arrive in bursts. One organization's merge-time burst is 10,000 jobs, and a small team has 5 waiting behind it. Who runs first, how does a job reach a machine, and how many machines does the pool need?

A concept that helps here: Job scheduling and delayed workA concept that helps here: Backpressure and load shedding

A runner that dies mid-job. A machine running a deploy job goes silent halfway through. What notices, what runs again, and what stops the same machine from finishing the deploy later under a lease that has already lapsed?

A concept that helps here: Distributed locks and leasesA concept that helps here: Heartbeats and presenceA concept that helps here: Idempotency keys

Logs, artifacts and caches. A developer watches a build's log live while a thousand other jobs write theirs. How does a line get from the runner to the screen, where does it end up, and how do caches decide a saved result is safe to reuse?

A concept that helps here: Blob storage and presigned uploadsA concept that helps here: Content addressing and deduplicationA concept that helps here: WebSockets, server-sent events and polling

Running untrusted code. A stranger's pull request against a public repository triggers its workflow. How does that job run without reaching another tenant's secrets or files, and how does a legitimate job get the credentials it needs?

A concept that helps here: Sandboxing untrusted codeA concept that helps here: Multi-tenant isolationA concept that helps here: Authorization models

Concepts this guide uses

Sources

  1. GitHub docs: Actions limits
  2. GitHub docs: Self-hosted runners reference
  3. GitHub blog: How GitHub runs its own CI on Actions larger runners
  4. GitHub blog: Let's talk about GitHub Actions
  5. Bazel docs: Remote caching
  6. GitHub docs: Using GitHub-hosted runners
  7. GitHub docs: Choosing the runner for a job
  8. GitHub docs: Understanding GitHub Actions
  9. GitHub docs: Events that trigger workflows
  10. GitHub docs: OpenID Connect
  11. GitHub docs: Secure use reference
  12. GitHub docs: Best practices for using webhooks
  13. GitHub docs: Webhook events and payloads
  14. Martin Kleppmann: How to do distributed locking
  15. GitHub docs: Dependency caching reference
Sign in to report a mistake