Sheltie

Sheltie is a parallel merge engine for agent-generated pull requests. It runs as a GitHub App next to your existing CI.

Coding agents open more pull requests than a one-at-a-time merge queue can land. Sheltie computes a footprint for every pull request: the build targets it writes and the targets it affects. Pull requests with independent footprints land in parallel. Conflicting ones are built together speculatively or resolved in escalating tiers:

1. structural merge (mergiraf);
2. regenerating derived files such as lockfiles;
3. merge rules you declare;
4. re-deriving the change with Claude Code, from the pull request itself, checked against the tests it adds.

Before work starts, Sheltie can also keep agents off overlapping issues.

How a pull request lands

1. An agent opens a pull request; GitHub notifies Sheltie.
2. An adapter for your build tool (Cargo, Go, JavaScript/TypeScript, Python, Maven or Gradle) runs on your CI runner and uploads the footprint.
3. Once the pull request is approved and its checks are green, the scheduler decides each epoch: land now, verify on the current head, build speculatively with others, or merge with the merge engine.
4. The committer checks that nothing that landed meanwhile touches the change, then fast-forwards the default branch. It never forces.
5. Sweeps watch the default branch’s CI; a red commit is bisected and reverted automatically.

Builds, tests and agent jobs run on your runners with your API keys. The control plane only does set arithmetic and talks to GitHub.

Status

Sheltie is early software (0.x). Start in shadow mode, where Sheltie only predicts what it would do and reports how often it would have been wrong, before letting it merge.

Supported build tools: Cargo. sbt, Mill, Gradle, Bazel and Maven are planned.

Next

• Self-hosting: run your own Sheltie.
• Workflows: what to add to your repositories.

Self-hosting

You run the control plane (one small container and Postgres) and register your own GitHub App. Plan for about 30 minutes.

What you need

• A Linux host with Docker (Compose v2), reachable from the internet over HTTPS at a host name of yours, for example sheltie.example.com. GitHub sends webhooks there and your CI uploads footprints there.
• A machine with a browser, Python 3 and a clone of this repository, to register the App.
• Owner or admin rights on the GitHub account or organization whose repositories Sheltie will manage.

1. Register the GitHub App

On the machine with the browser, in a clone of the repository:

scripts/register-github-app.py --url https://sheltie.example.com
# add --org your-org to register it under an organization

Confirm the App on GitHub when the browser opens. The script writes:

• sheltie-github-app.pem: the App’s private key (mode 600, git-ignored);
• .env: the App id and webhook secret, a random Postgres password, SHELTIE_PUBLIC_HOSTS and SHELTIE_UID.

No secret is printed.

2. Start Sheltie

On the host, in a clone of the repository, put the two files from step 1 in the repository root. If you registered the App on another machine, set SHELTIE_UID in .env to the output of id -u on the host: the server runs as that user so it can read the mode-600 key.

docker compose up -d --build
curl -s localhost:8080/healthz      # {"status":"ok"}

--build builds the image from source (a few minutes). Once releases are published you can use ghcr.io/softberries/sheltie instead: set SHELTIE_IMAGE in .env and drop --build.

The server listens on 127.0.0.1:8080. Put a TLS-terminating reverse proxy in front of it. With Caddy, which gets certificates automatically:

sheltie.example.com {
    reverse_proxy 127.0.0.1:8080
}

Check from outside: curl https://sheltie.example.com/healthz.

3. Install the App

Open the App’s page (the script printed its URL), choose Install, and select the repositories Sheltie should manage. Each repository starts in shadow mode: Sheltie posts a Sheltie check on every pull request with what it would do, but merges nothing.

4. Add the workflows

In each repository, add the footprint workflow so Sheltie learns what every pull request touches. See Workflows for the file to add, and for the optional merge-engine and replay workflows.

Open a pull request: the Sheltie check appears within seconds, and shows the footprint once the workflow has run.

5. Let Sheltie merge

After a few days in shadow mode, look at how often Sheltie’s predictions would have been wrong:

docker compose exec sheltie sheltie-server shadow-report --repo your-org/your-repo

Before switching a repository to merge mode:

• CI must run on pushes to branches under sheltie/ (Sheltie verifies changes on sheltie/verify/*, sheltie/spec/* and sheltie/bisect/*). Most workflows that run on: push already do.
• If the default branch is protected, add the App to the ruleset’s bypass list: Sheltie lands changes by fast-forwarding the branch. It never force-pushes.

Then:

docker compose exec sheltie sheltie-server repo set-mode --repo your-org/your-repo --mode merge

Switch back at any time with --mode shadow. To turn on the merge engine for a repository that has the merge workflow: sheltie-server repo set-merge-engine --repo your-org/your-repo --state on.

Dashboard

Open https://sheltie.example.com/dashboard and sign in with GitHub. You see the repositories where the App is installed and you have access: the queue, what landed and how it was proven, pull requests returned to their authors, agents’ claimed issues, how much time Sheltie saved and what replays cost (see Dashboard).

The registration script sets it up (SHELTIE_PUBLIC_URL and the App’s OAuth client id and secret). For an App registered before, add https://sheltie.example.com/auth/callback as a callback URL in the App’s settings, generate a client secret there, and set the three variables (see Configuration).

Kubernetes

A Helm chart is in deploy/helm/sheltie. Postgres is not included: use a managed database or an operator such as CloudNativePG, and put the connection URL in a Secret.

kubectl create secret generic sheltie-database --from-literal=url='postgres://…'
kubectl create secret generic sheltie-github \
  --from-literal=app-id=123456 --from-literal=webhook-secret='…' \
  --from-file=private-key.pem=sheltie-github-app.pem
helm install sheltie deploy/helm/sheltie \
  --set ingress.enabled=true --set ingress.hosts[0].host=sheltie.example.com

Take the App id and webhook secret from the .env the registration script wrote. See deploy/helm/sheltie/values.yaml for all settings.

Operating

• Upgrades: pull or build the new image and restart; migrations run on startup.
• Logs go to stderr; set RUST_LOG (for example info,sheltie_server=debug) for more.
• Backups: back up Postgres. Everything else can be rebuilt from GitHub.
• sheltie-server stats prints anonymous counts (no names) if you want to share how you use Sheltie. Sheltie sends no telemetry.

Configuration

sheltie-server reads its settings from environment variables, and from a .env file in the working directory if there is one.

Set all three SHELTIE_GITHUB_APP_ID, …_PRIVATE_KEY_PATH and …_WEBHOOK_SECRET, or none of them; without them the server serves only /healthz. Likewise set all three of SHELTIE_PUBLIC_URL and the client id and secret, or none (no dashboard).

compose.yaml also reads SHELTIE_POSTGRES_PASSWORD, SHELTIE_UID, SHELTIE_PORT, SHELTIE_IMAGE and SHELTIE_GITHUB_KEY_FILE from .env.

Per repository

Each repository starts in shadow mode. Change it with the CLI, which uses the same DATABASE_URL:

sheltie-server repo set-mode --repo owner/name --mode merge         # or shadow
sheltie-server repo set-merge-engine --repo owner/name --state on   # or off
sheltie-server token create --repo owner/name --name agent-a        # agent token, printed once
sheltie-server token revoke --repo owner/name --name agent-a
sheltie-server shadow-report --repo owner/name --days 7
sheltie-server stats --days 30

Merge rules and replay are configured in the repository itself, in .sheltie/merge.toml on the default branch: see Merge rules.

GitHub App permissions

From deploy/github-app-manifest.json:

• Permissions:
  • Checks: write
  • Contents: write
  • Issues: write
  • Metadata: read
  • Pull requests: write
  • Commit statuses: read
• Events: check_run, check_suite, issues, pull_request, pull_request_review, push, status.

Workflows

Sheltie never runs your build in the control plane. Three reusable GitHub Actions workflows run on your runners and report to Sheltie, authenticated with GitHub’s OIDC tokens (no shared secret). Add them to the default branch of each repository Sheltie manages, replacing https://sheltie.example.com with your server’s URL.

Pin the workflows to a release, as below: @v0.6.0 selects the workflow, and sheltie-version downloads that release’s checksum-verified binaries instead of building them from source, which takes a few minutes per run. @main with no sheltie-version follows the latest code.

Footprints (required)

.github/workflows/sheltie-footprint.yml:

on:
  pull_request:
  push:
    branches: [main]     # your default branch
permissions:
  contents: read
  id-token: write        # lets the job request the OIDC token
jobs:
  footprint:
    uses: softberries/sheltie/.github/workflows/footprint.yml@v0.6.0
    with:
      sheltie-version: v0.6.0
      sheltie-url: https://sheltie.example.com

On pull requests it computes the footprint with the adapter for your build tool and uploads it. On pushes to the default branch it uploads the build graph, which predicts footprints of issues (see Coordinating agents).

The adapter is picked from the files at the repository root: Cargo.toml (Cargo), go.mod or go.work (Go), pom.xml (Maven), settings.gradle(.kts) or build.gradle(.kts) (Gradle), package.json (JavaScript/TypeScript), pyproject.toml, setup.py or setup.cfg (Python), in that order. For a repository with several, set it:

    with:
      sheltie-url: https://sheltie.example.com
      tool: node   # cargo, go, node, python, maven or gradle

Files outside every unit (crate, package, module) are assumed to affect the whole build, so a change to one conflicts with everything. Declare data files that do not, one glob per line:

    with:
      sheltie-url: https://sheltie.example.com
      plain: |
        config/**
        migrations/**

Two changes to the same plain file still conflict (and go to the merge engine).

Pull requests from forks

GitHub gives workflow runs for pull requests from forks a read-only token and no OIDC token, so their footprint cannot be uploaded. The workflow then skips the upload with a notice instead of failing, and the Sheltie check keeps waiting for the footprint: Sheltie never lands a fork’s pull request. Review and merge it by hand as usual. Once it is on the default branch, sweeps cover it like any other commit, and because its footprint is unknown, ready pull requests are rebuilt on top of it before they land. Fork pull requests are not replayed either.

Merge engine (optional)

Resolves conflicts with tiers 1–3 (see Merge rules). Sheltie starts it with a repository_dispatch event. .github/workflows/sheltie-merge.yml:

on:
  repository_dispatch:
    types: [sheltie-merge]
permissions:
  contents: write        # pushes the merge result to sheltie/merge/*
  id-token: write
  pull-requests: read
jobs:
  merge:
    uses: softberries/sheltie/.github/workflows/merge.yml@v0.6.0
    with:
      sheltie-version: v0.6.0
      sheltie-url: https://sheltie.example.com
      pr: ${{ github.event.client_payload.pr }}
      head: ${{ github.event.client_payload.head }}
      onto: ${{ github.event.client_payload.onto }}

Then turn it on: sheltie-server repo set-merge-engine --repo owner/name --state on.

Replay (optional)

Re-derives a change with Claude Code when it cannot be merged otherwise, from the pull request itself; nothing has to be added to it. Needs your ANTHROPIC_API_KEY as a repository secret; see Replay for the workflow and its settings.

Your CI

Sheltie relies on your existing CI:

• A pull request is ready to land when it is approved (or needs no review) and all its checks are green.
• In merge mode, CI must run on pushes to branches under sheltie/: changes are verified on sheltie/verify/*, speculative builds run on sheltie/spec/*, and bisection uses sheltie/bisect/*.
• Sweeps read CI results of the default branch’s commits, and sheltie/verified points at the newest green one. Deploy from it if you want only verified commits.

Build tools

Sheltie learns what a pull request touches from an adapter for your build tool. The adapter runs on your CI runner (the footprint workflow installs and runs it), reads the base and head trees straight from git, and describes each as a graph of units (crates, packages, modules) with their dependencies and a hash of their public interface. Two changes conflict only when one writes a unit the other writes or affects, where a change affects the dependents of a unit only if it changes that unit’s interface.

Bazel, sbt and Mill are planned.

How accurate it is

The Cargo adapter parses Rust and compares module interfaces exactly. The others read the build files and the sources without running the build tool or a compiler, and extract the public surface from declarations line by line. They are deliberately pessimistic:

• When a declaration is ambiguous, more of it counts as interface.
• A unit’s interface folds in the interfaces of the units it depends on, because its own API may expose their types. An interface change therefore reaches dependents of dependents.
• A change that adds a dependency reads the new dependency, so it meets a concurrent change to that dependency.

The cost is that some changes look like interface changes when they are not, so a few more pull requests are built together than strictly needed. Body-only changes still stay inside their unit.

What no adapter sees: code loaded at run time by name (reflection, dynamic imports, plugin systems), and files read by tests or the build that are not part of a unit’s sources. Files inside a unit’s directory that are not its sources make the footprint leaky: Sheltie then widens it to everything that reads the unit.

Workspace-wide files (root manifests, toolchain versions, shared build logic) and dependency lockfiles affect every unit.

The command line

Every adapter has the same commands:

sheltie-go footprint --base origin/main --head HEAD   # the footprint, as JSON
sheltie-go graph --rev HEAD                           # the build graph, as JSON
sheltie-go roi --ci-minutes 12                        # the ROI report from history

--ignore and --plain add globs for files outside the build (see Workflows); roi is described in ROI report. All adapters come with every release.

Go

sheltie-go computes footprints for Go repositories. It reads everything straight from git: it does not need the go toolchain, a module cache or network access, and it never runs your code.

Units

Sheltie schedules Go code by package:

• Every go.mod is a module; its path comes from the module directive. A repository may hold several modules, and a nested module’s directories do not belong to the outer one.
• Every directory with .go files under a module root is a package, identified by its import path, e.g. example.com/shop/pricing. Directories with only test files are packages too.
• A package owns the .go files directly in its directory, test files included. Subdirectories are their own packages.
• Like the go command, Sheltie skips testdata directories and directories whose name starts with . or _.

A package reads every package of the repository it imports, from any of its files, test files included. Imports of other modules (the standard library, dependencies) are covered by go.mod and go.sum below.

Two changes to the same package always conflict, even in different files. A change to a package’s body never reaches the packages that import it; a change to its interface does.

The interface

A package’s interface is what importers can see. Sheltie hashes, from its non-test files:

• the package name and //go:build constraints;
• exported functions and methods, signature only (the body does not count). Methods count whatever their receiver, since an unexported type’s methods are reachable through interfaces and embedding;
• exported types, whole: every field (unexported ones too), method and type parameter;
• exported constants and variables, with their values. A const ( … ) group with an exported name counts as a whole, because iota and implicit repetition tie its lines together;
• unexported types and constants that any of the above mention, such as an embedded unexported struct or a constant an exported one is computed from.

Comments and formatting do not count (gofmt, rewrapping a parameter list, rewording a doc comment). Moving a declaration between files of the package does not count either.

A package’s interface also folds in the interfaces of the packages it imports, because an exported signature may use their types: an interface change in pricing reaches cart, which imports it, and cmd/shop, which imports cart. A change that adds an import reads the imported package.

The rule throughout: when unsure, include more. An extra interface change only widens a footprint and costs some parallelism; a missed one would hide a real conflict.

Workspace files

When the repository root is itself a package (a main.go next to go.mod), files outside every other package that are neither ignored nor plain (.github/**, lint configuration) count as that package’s data, and make footprints leaky.

Leaks

A package is not hermetic, and footprints that read it are marked leaky, when:

• it uses cgo (import "C"): C sources and headers are outside the package graph;
• it has a //go:embed pattern that cannot be resolved (malformed, or matching no file in the tree): the build would fail, so Sheltie claims nothing about it.

Files a package embeds are resolved from the tree with the go command’s rules (globs relative to the package directory, directories embedded recursively without . and _ names unless the pattern starts with all:, never files of another module). They belong to the package: changing one is a change to that package, and the package stays hermetic.

A changed file that no package owns is also leaky: inside a package’s directory (data files, testdata/, assembly or C files) it becomes <package>/files, which the package reads; elsewhere it is assumed to affect the whole build unless declared plain. Markdown, docs/ and licence files are ignored.

What it does not claim

• No type checking. Sheltie reads declarations textually; it does not resolve names across packages, so it cannot tell which part of an interface an importer actually uses. Any interface change reaches every importer.
• Build constraints are not evaluated: all files of a package count, whatever GOOS, GOARCH and tags they target.
• Unexported functions are never part of the interface, even when an exported variable is initialised by calling one.
• replace directives pointing outside the repository, and versions of external modules, are only seen as a change to go.mod or go.sum.
• Generated code counts as source: regenerate it in the same change.

Command line

# The footprint of HEAD against the merge base with origin/main, as JSON.
sheltie-go footprint --base origin/main --head HEAD

# Write it to a file, declare data files that do not affect the build.
sheltie-go footprint --base origin/main --out footprint.json --plain 'config/**'

# The build graph Sheltie sees at one revision.
sheltie-go graph --rev HEAD

# What Sheltie would have saved on this repository (see ROI report).
sheltie-go roi --ci-minutes 12

--ignore <glob> adds files that never affect the build. roi takes the same options as sheltie-cargo roi (see ROI report).

Build it with cargo install --git https://github.com/softberries/sheltie sheltie-go.

JavaScript and TypeScript

sheltie-node computes footprints for JavaScript and TypeScript repositories: npm, Yarn, pnpm and Bun workspaces, Lerna and Nx. It reads everything from git, so the runner needs git but not Node.js, a package manager or node_modules.

Supported layouts

sheltie-node finds the packages of a revision, in this order:

A directory matching a workspace glob that has a package.json is a package. Globs work as in the package managers: packages/* matches one level, packages/** any depth, and !pattern excludes. Directories under node_modules/ never count.

With none of these, a root package.json makes the whole repository one package.

Packages

Each package (a unit) is one target in the build graph:

• Name: the name in its package.json, else the Nx project name in project.json, else its directory.
• Files: everything under its directory except node_modules/, dist/, build/, out/, coverage/, .next/, .turbo/, nested packages, documentation (*.md) and lint or format configuration.
• Dependencies: workspace packages named in dependencies, devDependencies, peerDependencies or optionalDependencies (any version, including workspace:*); Nx implicitDependencies (except !excluded); and workspace packages its sources import, by package name (@acme/ui, @acme/ui/button), by a paths alias from the root tsconfig.base.json or tsconfig.json, or by a relative path into another package’s directory.
• Tests: a change to a package runs its tests, and those of every package it affects.

The public interface

A change to a package affects the packages that depend on it only if it changes the package’s public interface. The interface is:

1. the package.json fields name, version, main, module, types, typings, exports, bin, type, dependencies and peerDependencies;
2. from its sources (.ts, .tsx, .mts, .cts, .js, .jsx, .mjs, .cjs), except tests (*.test.*, *.spec.*, and anything under __tests__/, test/ or tests/):
   • every .d.ts file;
   • every top-level export statement, including export * from, export { a } from and export default;
   • every top-level type, interface, enum, declare and class, exported or not, because they can appear in exported signatures;
   • declarations exported by name (export { format }, module.exports = { format }) and CommonJS module.exports / exports.x statements.

Comments and formatting do not count. Of functions, arrow functions assigned to a const, and class methods, only the signature counts, when the body can be told apart from it: a change inside a function body does not reach dependents. When it cannot (for example export const f = memo((props) => { … })), the whole statement counts. Objects of functions (export const api = { get() { … } }) also count whole.

A package’s interface also folds in the interfaces of the workspace packages it depends on, because its exports may use their types: an interface change in @acme/utils reaches @acme/ui, which depends on it, and the apps that depend on @acme/ui. A change that adds a dependency reads the new dependency.

The rule throughout is to include more when unsure. An extra line in the interface only makes a footprint wider; a missing one would hide a real conflict.

Workspace files

Lint and format configuration (.eslintrc*, eslint.config.*, .prettierrc*, prettier.config.*, .prettierignore, .eslintignore, .editorconfig, at any depth) is tracked, so two changes to the same file conflict, but it does not affect any package. Documentation (*.md, docs/) is ignored.

Any other file outside every package, such as scripts/release.mjs or a root jest.config.js, is assumed to affect the whole build: the footprint is leaky and the change conflicts with everything. Declare data files that do not affect the build with --plain (or plain: in the footprint workflow).

Leaks

A footprint is leaky when the build may depend on more than the files Sheltie sees:

• a package whose package.json has a preinstall, install, postinstall or prepare script (install script): it can download or generate anything;
• a package whose package.json is not valid JSON;
• a changed file outside every package (see above).

What it does not claim

• It does not run TypeScript. The interface is read line by line; an unusual construct is counted whole rather than missed, and a file whose brackets do not balance (for example because of a regular expression containing {) counts whole.
• Data files inside a package (src/fixtures/orders.json) belong to the package but are not part of its interface, even when a module re-exports them.
• A package’s own tsconfig.json, bundler and test configuration belong to the package but are not part of its interface.
• In a workspace, install scripts in the root package.json (such as prepare: husky) do not make footprints leaky; changes to the root package.json affect every package anyway. (Without workspaces the root package.json is the package’s own, and its install scripts count.)
• The order of conditions in exports is not compared, only their content.
• Lockfiles count as a whole: any lockfile change affects every package.

Command line

sheltie-node has the same commands as sheltie-cargo. Run them in a checkout with history:

# The footprint of HEAD against main, as JSON
sheltie-node footprint --base origin/main

# The build graph of a revision, as JSON
sheltie-node graph --rev HEAD

# What Sheltie would have saved over the last 90 days (see the ROI report)
sheltie-node roi --ci-minutes 12

footprint takes --head (default HEAD), --repo, --out, and repeatable --ignore and --plain globs. roi takes the options listed in ROI report.

Python

sheltie-python computes footprints for Python repositories. It reads the repository straight from git and parses the source itself: it needs no Python interpreter, no virtual environment and no installed dependencies, so it runs in seconds on any CI runner.

Supported layouts

• uv workspaces: the members listed under [tool.uv.workspace] members (globs, minus exclude) in a pyproject.toml.
• Projects: every directory with a pyproject.toml, setup.py or setup.cfg, at any depth (monorepos with several projects, Poetry, PDM, Hatch and setuptools alike).
• src layout and flat layout: a project’s source root is its src/ directory if it has one, else the project directory itself. A setuptools package-dir = {"" = "lib"} or [tool.setuptools.packages.find] where = ["lib"] in pyproject.toml overrides this.
• No project files at all: the repository root is the source root.

.venv/, venv/, node_modules/, build/, dist/, __pycache__/, .tox/ and every other dot-directory are skipped.

Units

Sheltie’s targets for Python are import packages, named the way you import them:

Test modules inside a package belong to the package. If two projects contain a package with the same name, the ids get the directory appended (utils@a/src/utils), and imports of the name read both.

A unit reads the units its files import: import a.b.c, import a.b as x, from a.b import c, d, parenthesised multi-line imports, and relative imports (from . import x, from ..pkg import y, resolved against the importing file’s package). Each import resolves to the deepest unit it names (a.b.c, else a.b, else a); from a.b import c also reads a.b.c when that is a unit. Imports count wherever they are: inside functions, try blocks and if TYPE_CHECKING:. A test directory also reads the test directories above it, whose conftest.py fixtures it uses.

Interfaces

A change only reaches the units that import a package when the package’s public interface changes. The interface is read from the package’s modules (test modules excluded), with comments removed and formatting ignored:

• public functions (def and async def whose name does not start with _): decorators and the signature with parameters, defaults and annotations, not the body;
• public classes: decorators, bases, every method signature except _private ones (dunders such as __init__ count), class-level attributes and annotations, and nested public classes;
• module-level assignments to public names, type aliases (type T = …) and __all__;
• in __init__.py, every import (they are re-exports). A package that re-exports another unit, and any module with from x import *, includes that unit’s interface in its own;
• statements inside module-level if, try and with blocks, and the block headers.

So editing a function body, a private helper (_round, even its signature), a docstring or a comment stays inside the package; changing a public signature, a public constant, a class member or a re-export reaches every unit importing the package, and the units importing those: a unit’s interface also folds in the interfaces of the units it imports, because its signatures may use their types. A change that adds an import reads the imported unit. When the adapter is unsure, it counts a statement as interface: a wider footprint only costs parallelism, a missed interface change could hide a conflict.

A unit’s tests are its own id plus the test directories that import it, so a body change in shop_core also lists tests:packages/core/tests.

Configuration, lockfiles and leaks

• $workspace owns pyproject.toml, setup.py and setup.cfg of every project, tox.ini, noxfile.py, pytest.ini, Pipfile, .python-version and the root conftest.py. Every unit reads it, so changing any of them conflicts with every change.
• $lockfile owns uv.lock, poetry.lock, Pipfile.lock, pdm.lock, requirements*.txt and constraints*.txt.
• Linter and formatter settings (.flake8, ruff.toml, .ruff.toml, mypy.ini, .pylintrc, .pre-commit-config.yaml) are plain files: two changes to one conflict, but they affect no unit.
• A project whose setup.py mentions ext_modules or cmdclass builds extensions or runs custom build steps, so its units are not hermetic: footprints reading them are leaky (setup.py builds extensions).
• Other files in a package directory (templates, JSON, .pyx sources) are owned by no unit: a change to one is attributed to the package and is leaky. Files outside every unit are leaky unless they are plain.

What it does not claim

The adapter sees imports that are written down. It does not see:

• dynamic imports: importlib.import_module(name), __import__, plugin entry points, Django INSTALLED_APPS strings and similar registries;
• names reached through sys.path manipulation, .pth files or namespace packages split across several source roots under different layouts;
• source roots configured in setup.cfg (package_dir), Poetry packages = [{ from = … }] or Hatch build targets (only src/ and the setuptools keys above are recognised);
• the effect of a body change on behaviour that dependents observe (a function returning a different value): like for every adapter, the merge check and sweeps cover that by running tests;
• f-strings that reuse their own quote character inside {…} (allowed since Python 3.12), which can confuse statement splitting on that line.

Third-party packages are not units: a dependency upgrade shows up as a lockfile or pyproject.toml change.

Command line

sheltie-python has the same commands as every adapter:

# The footprint of HEAD against main, as JSON (what the footprint workflow uploads).
sheltie-python footprint --base origin/main

# The build graph Sheltie sees for one revision.
sheltie-python graph --rev HEAD

# Replay the last 90 days of merges and estimate what Sheltie would have saved.
sheltie-python roi --ci-minutes 12

footprint takes --ignore <glob> and --plain <glob> (repeatable) for files that never affect the build and for data files outside it; roi takes the options described in ROI report.

Maven

sheltie-maven computes footprints for Maven repositories. It reads the POMs and the Java and Kotlin sources straight from git: it needs no mvn, no JDK and no network, and it never runs your build. It has the same commands and output as sheltie-cargo.

cargo install --git https://github.com/softberries/sheltie sheltie-maven

Units

Every module of every reactor in the repository is a unit: the root pom.xml, and any other pom.xml outside src/ and target/ (a second reactor such as guava’s android/):

• Modules are found through <modules> (including those in <profiles>), recursively. A <module> can name a directory or a POM file. A <parent> found by its relativePath (default ../pom.xml) is a unit too, even if no <modules> lists it.
• Identity: groupId:artifactId, for example com.acme.shop:core. A missing groupId is inherited from <parent>. When two reactors have a module with the same coordinates, both ids get the directory (com.google.guava:guava@android/guava), and a dependency resolves to the module of its own reactor.
• Files: the module’s POM and everything under its src/ (main and test sources, resources). Nested modules own their own files.
• Dependencies: a module reads the modules named by its <dependencies>, its build plugins, their <dependencies> and annotationProcessorPaths, including those of profiles, and its parent. <dependencyManagement> and <pluginManagement> only declare versions and create no edge. ${project.groupId}, ${project.version} and properties from <properties> (the parent’s included) are resolved.
• Workspace configuration: .mvn/**, mvnw and mvnw.cmd. A change to them affects every module.

Plain files (checkstyle*.xml anywhere, .editorconfig) are tracked, but do not make a footprint leaky. Anything else outside a module is assumed to affect the whole build; declare data files with --plain (see Workflows).

Interfaces

A change that does not alter a module’s interface affects only that module. The interface is:

• the public surface of src/main/java and src/main/kotlin;
• the whole POM, without comments and with whitespace normalised. Any POM change (a new dependency, a plugin setting) therefore reaches the module’s dependents: pessimistic, because dependencies are transitive;
• the contents of src/main/resources/META-INF/ (service registrations and similar files loaded by dependents at run time);
• the test sources too, when another module depends on its test-jar.

The public surface of a Java file is its package line and every public or protected type with its header (type parameters, extends, implements, record components). In their bodies: public and protected fields (whole, with initialisers, because javac inlines constants), method and constructor signatures, enum constants, every non-private member of interfaces and annotation types, and nested types by the same rules. Annotations count; method bodies, comments, formatting, imports, and private and package-private declarations do not. A module-info.java counts whole.

For Kotlin, everything that is not private or internal counts (@PublishedApi internal does). Functions with a block body count by signature. Expression-bodied and inline functions, properties with initialisers and type aliases count whole: an inferred type, a constant or an inlined body depends on them.

When in doubt the adapter includes more: an extra declaration only makes footprints wider, a missed one would hide a conflict.

A module’s interface also folds in the interfaces of the modules it depends on, because its public signatures may use their types: an interface change in one module reaches its dependents and theirs. A change that adds a dependency reads the new dependency.

Leaks

A module is not hermetic when its build runs code Sheltie cannot see: exec-maven-plugin, maven-antrun-plugin or build-helper-maven-plugin in its <build><plugins> or its parent’s (inherited). Footprints that read such a module are leaky and give the reason, for example com.acme.shop:api: build plugin runs code (exec-maven-plugin).

What it does not claim

• Reflection, dependency injection, service loading by name: code that finds classes or members at run time (Spring, Jackson, Class.forName) can depend on a private or package-private declaration. Such changes look local.
• Annotation processors and compiler plugins: generated code (Lombok, MapStruct, Dagger, Immutables) is not seen, so a change to an annotated private field can change a generated public API unnoticed. A processor built in the same repository is a dependency of the modules that use it.
• Split packages: package-private declarations are ignored, though another module (often its tests) can use them if it declares the same package.
• Imports: changing which class a simple name in a signature refers to only through an import is not seen.
• Resolution: dependency versions, BOMs, <scope> and <optional> are not resolved. Every POM change counts as an interface change, and changes to external dependencies are not seen beyond the POM text.
• Other layouts: src/main/{java,kotlin}, Kotlin Multiplatform src/<x>Main and a <sourceDirectory> / <testSourceDirectory> set in the module or a parent feed the interface. Generated sources and Groovy or Scala sources are not read.
• Other plugins: only the three plugins above are flagged as leaky. Any plugin can read files outside the module; if yours do, declare the files or treat them as workspace configuration.

Command line

# The footprint of HEAD against main, as JSON
sheltie-maven footprint --base origin/main

# The build graph of a revision, as JSON
sheltie-maven graph --rev HEAD

# What Sheltie would have saved over the last 90 days (see the ROI report)
sheltie-maven roi --ci-minutes 15

footprint takes --head, --repo, --out and repeatable --ignore and --plain globs. roi takes the options in ROI report; with --exact it computes each landing’s footprint with the adapter, which needs only git (no Maven).

Gradle

sheltie-gradle computes footprints for Gradle repositories, with Groovy or Kotlin DSL build scripts and Java or Kotlin sources. It reads the scripts and sources straight from git: it needs no Gradle, no JDK and no network, and it never evaluates a script. It has the same commands and output as sheltie-cargo.

cargo install --git https://github.com/softberries/sheltie sheltie-gradle

Units

Every project in settings.gradle or settings.gradle.kts is a unit:

• Projects come from include statements in their common forms: include ':a', ':b:c', include(":a", ":b"), include("a"), spread over several lines. :a:b lives in a/b, unless the settings set project(":a:b").projectDir = file("dir") (or new File(settingsDir, "dir")). Parents of nested projects (:a for :a:b) are projects too. The root project : is a unit if it has a src/ directory.
• Identity: the Gradle path, for example :core or :libs:io.
• Files: the project’s build.gradle(.kts), its gradle.properties and everything under its src/.
• Dependencies: a project reads the projects its build script names: project(":x"), project(':x'), project(path: ":x"), project(path = ":x"), and type-safe accessors such as projects.core or projects.libs.sharedUtils (kebab- and snake-case names become camelCase). The whole script is searched, not only dependencies { }.
• Workspace configuration: the settings script, the root build script, gradle.properties, version catalogs (gradle/*.versions.toml), the wrapper (gradle/wrapper/**, gradlew, gradlew.bat), convention plugins in buildSrc/ and build-logic/, and builds included with includeBuild. A change to them affects every project.
• Lockfiles: gradle.lockfile (in any project) and gradle/verification-metadata.xml. A change to them affects every project.

Plain files (.editorconfig, detekt.yml anywhere) are tracked, but do not make a footprint leaky. Anything else outside a project is assumed to affect the whole build; declare data files with --plain (see Workflows).

Interfaces

A change that does not alter a project’s interface affects only that project. The interface is:

• the public surface of the main sources: src/main/{java,kotlin}, Kotlin Multiplatform src/<x>Main/{java,kotlin} and src/testFixtures/{java,kotlin} (consumed with testFixtures(project(":x")));
• the project’s build script and gradle.properties, without comments and with whitespace normalised. Any change to them (a new dependency, a plugin setting) therefore reaches the project’s dependents: pessimistic, because api dependencies are transitive;
• the contents of src/main/resources/META-INF/ (service registrations and similar files loaded by dependents at run time).

The public surface of a Kotlin file is its package line (with file annotations such as @file:JvmName) and every declaration that is not private or internal (@PublishedApi internal counts). Classes, interfaces and objects count by their header and the surface of their body. Functions with a block body count by signature. Expression-bodied and inline functions, properties with initialisers and type aliases count whole: an inferred type, a constant or an inlined body depends on them.

For Java, the package line and every public or protected type with its header count; in their bodies, public and protected fields (whole, with initialisers), method and constructor signatures, enum constants, every non-private member of interfaces and annotation types, and nested types by the same rules. Annotations count; method bodies, comments, formatting, imports, and private and package-private declarations do not.

When in doubt the adapter includes more: an extra declaration only makes footprints wider, a missed one would hide a conflict.

A project’s interface also folds in the interfaces of the projects it depends on, because its public signatures may use their types: an interface change in one project reaches its dependents and theirs. A change that adds a dependency reads the new dependency.

Leaks

Projects are hermetic by default. Files a build script reads from outside its project (apply from: "../shared.gradle", file("../data")) are outside every unit, so changes to them make footprints leaky unless they are under a project’s src/ or workspace configuration.

What it does not claim

• Script evaluation: settings are not run. Projects included in ways the parser cannot follow (helper functions, loops) are still found when their directory has a src/ and a build script, and named by Gradle’s convention (a/b is :a:b); a project whose directory differs from its name that way is not. A build script may be build.gradle(.kts), named after its directory (core/core.gradle.kts), or the only script in the directory. Dependencies added by convention plugins or subprojects { } blocks are not seen; convention plugins and the root build script are workspace configuration, so changing them still affects every project.
• Reflection, dependency injection, service loading by name: code that finds classes or members at run time can depend on a private or internal declaration. Such changes look local.
• Annotation processors and compiler plugins: code generated by kapt, KSP or Java annotation processors (Room, Dagger, Moshi, Lombok) is not seen, so a change to an annotated private declaration can change a generated public API unnoticed.
• Android: build variants, flavours and source sets other than main (such as src/debug), res/ and AndroidManifest.xml are owned by the project but do not feed its interface; resource IDs shared between modules are not compared.
• Imports: changing which class a simple name in a signature refers to only through an import is not seen.
• Resolution: dependency versions and configurations are not resolved. Every build script change counts as an interface change.

Command line

# The footprint of HEAD against main, as JSON
sheltie-gradle footprint --base origin/main

# The build graph of a revision, as JSON
sheltie-gradle graph --rev HEAD

# What Sheltie would have saved over the last 90 days (see the ROI report)
sheltie-gradle roi --ci-minutes 15

footprint takes --head, --repo, --out and repeatable --ignore and --plain globs. roi takes the options in ROI report; with --exact it computes each landing’s footprint with the adapter, which needs only git (no Gradle).

Merge rules

When a pull request conflicts with what landed and the merge engine is on (see Workflows), the sheltie-merge workflow merges it on your runner:

1. Structural merge. Conflicted source files go to mergiraf, which merges by syntax tree. It is GPL-3.0 and only ever runs as a separate program.
2. Derived files are regenerated instead of merged.
3. Declared rules merge files that have a known shape.

The result is verified like any other change before it lands. Files no tier can resolve return the pull request to its author, or go to intent replay if it is enabled.

Rules live in .sheltie/merge.toml on the default branch. Sheltie never reads them from the pull request being merged. They are parsed strictly: unknown keys, invalid globs or regular expressions, files over 64 KB and more than 100 rules are rejected.

[[derived]]          # regenerate instead of merging
paths = ["Cargo.lock"]
regenerate = "cargo update -w --offline"

[[set]]              # order-insensitive lines: union of both sides
paths = ["config/feature_flags.txt"]

[[sequence]]         # numbered files: renumber the later side on collision
paths = ["migrations/*.sql"]
key = "^(\\d+)_"

[[derived]]

Files generated from others, such as lockfiles. The engine takes one side, then runs regenerate (one line, at most 500 characters, run with sh -c in the checkout). This also happens when both sides changed the file without a textual conflict. A command shared by several conflicted files runs once.

[[set]]

Files whose lines form an unordered set (feature flags, allow lists, CODEOWNERS-style lists). Conflicts are resolved with a three-way union of lines: lines either side added are kept (ours first), lines either side removed are dropped, and duplicate lines are removed.

[[sequence]]

Numbered files, such as database migrations, where both sides added a file with the same number. key is a regular expression whose first capture group is the number; the file the later side added is renumbered to the next free number.

Two changes that each add a file matching the same rule conflict even when their file names differ (the adapter gives them a shared seq: target), so they always go through the merge engine and never land with the same number.

When more than one rule matches a path, [[derived]] wins over [[set]], which wins over [[sequence]].

[replay]

Settings for tier 4, re-deriving a change with an agent: enabled, the test commands (test_command, and test_commands per language), the model and the limits. See Replay.

[estimates]

Assumptions for the dashboard’s time-saved estimate. Both are optional, in minutes (at most 1440). See Dashboard.

[estimates]
minutes_per_conflict = 15   # an author fixing a conflict the merge engine resolved
minutes_per_replay = 30     # an author redoing a change Sheltie re-derived

The server reads this file from the default branch every 10 minutes.

[[agents]]

Names the agents behind pull requests, for the agent scorecards. Each rule has a name, an optional model, and at least one condition; every condition given must match, and the first matching rule wins.

[[agents]]
name = "codex"
model = "gpt-5-codex"   # optional, shown next to the name
branch = "codex/"       # the head branch starts with this

[[agents]]
name = "claude"
label = "agent: claude" # the pull request has this label (case-insensitive)

[[agents]]
name = "release-bot"
author = "acme-release[bot]"  # the author's login (case-insensitive)

Without a matching rule, a pull request opened by a bot account counts as that bot (claude[bot] becomes claude), and pull requests opened by people are grouped as “People”. Names are at most 40 characters; at most 50 rules. The server reads this file from the default branch every 10 minutes, and the rules apply to past pull requests too.

Replay

When a pull request cannot be merged by tiers 1–3, or breaks CI once merged onto the new head (a semantic conflict), Sheltie can have Claude Code re-derive it on your runner, with your API key.

• 4a resolves the conflicted hunks.
• 4b re-implements the change on the new head.

A failed 4a is retried as 4b. After max_attempts failed attempts, the pull request returns to its author.

Replay works the same for every pull request, whoever wrote it: Claude Code, Codex, Copilot, Devin or a person. Nothing has to be added to the pull request.

What the agent works from

The goal comes from what every pull request already has:

• its diff;
• its title and description;
• its commit messages;
• the issue it closes (Fixes #12), as Sheltie last saw it;
• what the default branch changed in the same files since the pull request branched off.

This text is written outside the replay, so the agent is told to treat it as a description of the change, never as instructions. The agent’s limits (below) still apply, whatever the text says.

Acceptance tests, found automatically

A test the pull request adds or changes, that passes with the change and fails without it, shows what the change does. Sheltie finds these tests itself (fail-to-pass), in the replay job:

1. It reads the pull request’s diff for test functions it added or changed:

   Language	Test	Name passed to the command
   Rust	#[test], #[tokio::test] and other *::test attributes	function name
   Go	func TestX(t *testing.T) in _test.go	TestX
   Python	def test_* in test_*.py or *_test.py	pytest node id, path::Class::test_x
   JavaScript, TypeScript	it(...) and test(...) in *.test.* and *.spec.*	the test’s title
   Java, Kotlin	@Test methods	Class#method

   It takes at most 10.

2. It runs each one alone on the pull request’s head, where it must pass, and on the commit the pull request branched from, where it must fail. A test that does not exist there counts as failing, and is not run: some runners pass when a filter matches nothing.

3. The tests that pass with the change and fail without it are the acceptance tests. The replay result must pass them, and must still define them.

Each run is limited to 10 minutes and the whole step to 30. The check run lists the tests found, which were used, and why the others were not.

What this cannot find:

• refactors, documentation changes and pull requests that add no test;
• tests whose titles are built at run time, such as JavaScript template strings with ${};
• tests a framework cannot run alone by name.

A full rebuild (4b) with no acceptance test always needs a person to approve it, however similar it is to the original: CI on the merged result only shows that it builds. A 4a result needs no acceptance test; it keeps both sides of a textual conflict, and your CI checks it.

An explicit intent (optional)

An author can still describe the change in a fenced sheltie-intent block in the description. The agent then also gets the author’s task and plan, and the listed tests are added to the acceptance tests:

```sheltie-intent
{
  "task": "Add retries to the billing client",
  "plan": "Wrap send() in a loop with backoff; add a test",
  "acceptance": ["billing::client::tests::retries_three_times"]
}
```

• task is required, at most 8000 characters.
• plan is optional, at most 16000 characters.
• acceptance lists up to 50 test names, which may only contain letters, digits and _ : . - /. They run together with test_command.

Unknown fields are rejected, and an invalid block is ignored. Editing the description updates the intent; adding one to a pull request Sheltie returned retries it.

Turning it on

In .sheltie/merge.toml on the default branch (every key is optional except enabled; defaults shown):

[replay]
enabled = true
test_command = "cargo test --workspace -- {tests}"  # {tests} becomes the quoted test names
test_commands = { python = "pytest {tests}" }       # per language, overrides test_command
model = "sonnet"
max_turns = 30              # per run
max_budget_usd = 3.0        # per run, as estimated by Claude Code
daily_budget_usd = 20.0     # per repository, over the last 24 hours
similarity_threshold = 0.6  # below this, the result needs a human
max_attempts = 2            # per pull request head

The languages in test_commands are rust, go, python, javascript and java. Tests in a language without a command are skipped. The replay job does not set up your project, so a command may include its own setup:

test_commands = { javascript = "npm ci && npx vitest run -t {tests}" }

For Gradle, which selects Class.method, convert the name: ./gradlew test --tests "$(echo {tests} | tr '#' .)".

Then add the replay workflow, .github/workflows/sheltie-replay.yml:

on:
  repository_dispatch:
    types: [sheltie-replay]
permissions:
  contents: write        # the publish job pushes the result to sheltie/replay-result/*
  id-token: write        # authenticates the report to Sheltie
  pull-requests: read
jobs:
  replay:
    uses: softberries/sheltie/.github/workflows/replay.yml@v0.6.0
    with:
      sheltie-version: v0.6.0
      sheltie-url: https://sheltie.example.com
      pr: ${{ github.event.client_payload.pr }}
      head: ${{ github.event.client_payload.head }}
      onto: ${{ github.event.client_payload.onto }}
      mode: ${{ github.event.client_payload.mode }}
    secrets:
      anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}

What is accepted

After the agent runs, the replay client checks that:

• no conflict markers are left;
• only files from the original pull request and the conflicted files changed;
• the acceptance tests pass.

It also measures how similar the new diff is to the original (the share of changed lines they have in common). Then:

• Checks pass and similarity is at or above the threshold: Sheltie verifies the result like any merge (your CI runs on it), lands it, and labels the pull request sheltie: replayed.
• Similarity is below the threshold, or a 4b result has no acceptance test: the result is kept on sheltie/replay/<pr>-<head>, the pull request is labelled sheltie: needs re-approval, and it returns to its author.

Security

• Your API key stays in your repository’s secrets; Sheltie never sees or stores it.
• The agent runs with --bare (no repository hooks or MCP servers) and no prompts. It can only use file tools and Bash for the test commands; web tools are disabled. Turns and spend are capped.
• The pull request’s title, description and commits, and the linked issue, go into the agent’s prompt. Anyone who can write them can try to steer the agent. The limits above are what stops that: the agent can only edit the pull request’s own files, and the result is checked and verified before it lands.
• The agent and the tests run the pull request’s code. They do so in a job with a read-only token and no git credentials, and the tests do not get the API key. A separate job that runs no pull request code pushes the result.
• Pull requests from forks are not replayed.
• Settings come only from the default branch.

Coordinating agents

Sheltie can keep coding agents off each other’s toes before they write code. It predicts which build targets an issue will touch, and lets only one agent at a time work on issues that overlap. Pull requests are still serialized at merge time; this avoids the wasted work.

Setup

1. Let Sheltie see your build graph: call the reusable footprint workflow on pushes to the default branch too (see the header of .github/workflows/footprint.yml). Each push uploads the graph to POST /v1/graphs. Only runs for a push to the default branch are accepted.

2. The GitHub App needs Issues: write and the issues event (see deploy/github-app-manifest.json). Existing installations must accept the new permission.

3. For agents that call Sheltie, create a token per agent:

   sheltie-server token create --repo acme/widgets --name agent-a
   

   The token (shl_…) is printed once; Sheltie stores only its SHA-256. The name is who holds the agent’s leases. Revoke with sheltie-server token revoke --repo acme/widgets --name agent-a.

Agents with MCP

The MCP server is at /mcp (Streamable HTTP, stateless). For Claude Code:

claude mcp add --transport http sheltie https://sheltie.example.com/mcp \
  --header "Authorization: Bearer shl_…"

Tools:

Set SHELTIE_PUBLIC_HOSTS to the server’s public host name(s): the MCP endpoint rejects other Host headers (DNS-rebinding protection).

Agents assigned on GitHub

Agents that work from issue assignments (no MCP) are coordinated through the issue itself:

• Assigning an issue claims it for the assignee.
• If it overlaps an issue someone else holds, Sheltie labels it sheltie: blocked and comments which issue it waits for. When that work lands, Sheltie claims it for the assignee, removes the label and comments again.
• Unassigning releases the claim.
• New issues get a comment with their predicted footprint.

Sheltie cannot stop an agent that ignores the label; its pull request is still serialized at merge time.

REST

The same operations, with the same token: GET /v1/issues/safe, POST /v1/leases ({"issue": N}; 409 when blocked or taken), DELETE /v1/leases/{issue}, GET /v1/footprint?paths=a,b or ?issue=N.

How it decides

• Prediction. File paths, module::paths and `identifiers` in the issue’s title and description are matched against the default branch’s graph. Written targets are assumed to change their interfaces, so their direct readers count as affected. A new file belongs to the targets of its directory. If nothing matches, the footprint is unknown: the issue can be claimed but is never listed as safe.
• Leases. Two issues conflict when one writes what the other writes or reads. A claim is checked against all active leases atomically. Leases last SHELTIE_LEASE_TTL_HOURS (default 4) and renew when claimed again. They end on release, on expiry, when the issue closes, or when a pull request that closes it (Fixes #N) lands.

Dashboard

The dashboard at /dashboard shows, for every repository where the App is installed and you have access on GitHub:

• Time saved over the last 30 days (below).
• Activity: landings per day, and what is ready, in flight, landed and returned now.
• Parallelism: how many agents the repository can take at once before their changes start colliding (below).
• Agents: a scorecard per agent that opens pull requests (below).
• Agent spend: what replays cost, in estimated dollars and tokens (below).
• Queue: open pull requests, their state, footprint and what Sheltie will do next.
• Recently landed: each landing’s commit, how it was proven safe, and which tier resolved a conflict.
• Returned to their authors, with the reason, and agents at work (claimed issues).

Sign in with GitHub. Sheltie reads which repositories you can access when you sign in, so after installing the App on another repository, sign out and in again. The dashboard is read-only. Setting it up is part of Self-hosting.

Time saved

Sheltie saves two different kinds of time, and the dashboard shows them separately instead of adding them up.

Waiting time. How much sooner pull requests reached the default branch than they would have in a one-at-a-time merge queue, the usual alternative. Sheltie simulates that queue from the real times your pull requests became ready (approved, checks green): each one lands one CI run after the later of its ready time and the previous landing. The CI time is the median measured duration of CI on your default branch (from the first check run’s start to the last one’s end, Sheltie’s own check excluded). Sheltie’s own landing times are measured. The headline is the difference summed over the period; the panel also compares the median time from ready to landed. If Sheltie was slower, the panel says so.

Author time. Conflicts Sheltie resolved without the author: merge-engine resolutions (structural merges, regenerated files, declared rules) and changes re-derived by an agent. Each counts at an assumed number of minutes an author would have spent instead, 15 per conflict and 30 per re-derived change by default. Set your own in [estimates] in .sheltie/merge.toml. The dashboard always shows the assumption next to the number.

Also shown: changes landed through Sheltie, and bad commits it found and reverted.

What it does not claim:

• The comparison is with a one-at-a-time merge queue, not with merging by hand without a queue.
• Waiting time is wall-clock time until a change is on the default branch, not people’s working hours.
• With fewer than 10 landings in the period, or no CI time measured yet, there is no waiting estimate.
• In shadow mode nothing lands through Sheltie, so the panel shows the predicted saving: the time merges waited after Sheltie would have landed them.

Parallelism

How many agents can work on the repository at the same time while most of their changes still land on their own, without a combined build or the merge engine.

Sheltie estimates it from the footprints of the pull requests merged into the default branch in the last 90 days (at most the 2,000 most recent). It treats them as the kind of work an agent picks up next. With N agents each holding one open pull request, a change lands on its own when it is independent of the other N - 1. A change that conflicts with a share c of the history stays independent with probability (1 - c)^(N - 1). The panel shows the average over the history for 1, 2, 4, … 64 agents, and the headline is the largest number of agents for which at least 80% of changes land on their own.

Two changes conflict when one writes a target the other writes or affects, the same rule the scheduler uses. A conflict is not a failure: Sheltie builds the changes together or resolves them in tiers. It costs a CI run or a merge, though, and conflicts are where waiting time comes back.

The panel also names the targets involved in the most conflicting pairs. Those are the places where splitting a crate or a central file would raise the number.

What it does not claim:

• It assumes agents pick work independently of each other. Work that clusters (a refactoring and its follow-ups) collides more than the estimate says.
• Pull requests whose footprint was never uploaded are left out. With fewer than 20 footprints, there is no estimate.

Agents

One row per agent, for the pull requests opened in the last 30 days:

• PRs opened, and merged with the merge rate: merged against closed without merging. Open pull requests count in neither.
• Returned: Sheltie sent the pull request back to its author at least once (a conflict it could not resolve, or CI failing on the combined state), out of all pull requests.
• Reverted: a sweep found it broke the default branch and Sheltie reverted it, out of the merged ones. Reverts done by hand are not counted.
• Replayed: merged after Sheltie re-derived it, out of the merged ones.
• Review rounds: reviews requesting changes, per merged pull request.
• Time to merge: median time from Sheltie first seeing the pull request to its merge.
• Replay cost: what Sheltie’s replays of the agent’s pull requests cost, per merged pull request. It is not what the agent itself cost: Sheltie does not see that.

Which agent opened a pull request comes from [[agents]] rules in .sheltie/merge.toml: by author, branch prefix or label. Without a rule, bot accounts count as themselves and people are grouped as “People”, so the panel never ranks colleagues. Pull requests opened before Sheltie recorded authors show as “Not recorded”.

The same table is available as text, Markdown or JSON:

sheltie-server scorecard --repo acme/widgets --days 30

Agent spend

Only replay uses a model. Structural merges, regenerated files, declared rules, footprints and scheduling cost nothing. The panel shows, for the last 30 days:

• Estimated spend, per day and in total, with the number of replays, how many passed the checks, and the model most replays used.
• The last 24 hours against daily_budget_usd, when replay is enabled. That is the window the budget limits. The bar turns red from 80%.
• Tokens: input, output, and prompt-cache reads and writes, summed over the whole agent run, including subagents. Replays run by a replay workflow older than this feature report no tokens. The panel then says how many of the replays the counts cover.
• Per replay (average cost) and per hour saved: the spend divided by the author time saved in the same period (see Time saved).

The overview adds each repository’s spend to its card and the total to the summary.

The amounts are the agent’s own estimates (Claude Code’s total_cost_usd), not your bill. Replays run on your CI runners with your API key, so check your provider’s console for actual charges.

ROI report

Before installing anything, find out what Sheltie would have saved on your repository. sheltie-cargo roi replays the last 90 days of your default branch through Sheltie’s scheduler and compares it with a one-at-a-time merge queue. It runs locally in your checkout and sends nothing anywhere.

git clone https://github.com/acme/widgets && cd widgets
sheltie-cargo roi --ci-minutes 12

Sheltie ROI report · acme/widgets
412 landings on origin/main, 2026-07-20 to 2026-10-17

  9 d 2 h less waiting and 361 fewer CI runs than a one-at-a-time merge queue

                            One at a time        Sheltie
  Waiting, total                 11 d 6 h       2 d 4 h
  Waiting, median                  48 min           0 s
  Waiting, 90th percentile     3 h 10 min        24 min
  Extra CI runs                       412            51
…

(Illustrative output.) sheltie-cargo comes with every release, or build it with cargo install --git https://github.com/softberries/sheltie sheltie-cargo.

Options

To get the median CI time of a GitHub Actions workflow named CI on main:

gh run list --workflow CI --branch main --event push --status success --limit 50 \
  --json startedAt,updatedAt \
  --jq '[.[] | ((.updatedAt|fromdate) - (.startedAt|fromdate)) / 60] | sort | .[length/2|floor] | ceil'

How it works

The history. Every first-parent commit on the branch in the period is a landing: a merged pull request (merge or squash commit) or a direct push. Its change is the diff against its first parent. It counts as ready when it landed, which is the earliest it could have been.

What each change was tested on. A change’s own CI ran on the branch as it was when the pull request was last based. Sheltie rebuilds a change when something that landed after that touches what it reads, so the replay needs that base:

• for a merge commit, the merge base of its second parent (the pull request head) with the branch;

• for a squash commit whose subject ends in (#123), the merge base of refs/pull/123/head, if those refs have been fetched:

  git fetch origin '+refs/pull/*/head:refs/pull/*/head'
  

• otherwise (direct pushes, rebase merges, unfetched heads) it is unknown, and the change counts as tested on the branch as it was when it landed. That leaves out rebuilds Sheltie would have needed, so the report says how many changes it applies to.

The merge base is the oldest the tested branch can have been; CI that ran later on a merge ref saw a newer branch, so with a known base the replay errs towards more rebuilds.

Footprints. By default each landing’s footprint is predicted from its changed files against the build graph of the branch tip, the way Sheltie predicts footprints for agents. Prediction is pessimistic: every change is assumed to alter the interfaces of what it writes, so more changes conflict than really would. Files no target owns count as their own targets. With --exact, sheltie-cargo computes every landing’s footprint as CI would; that runs cargo metadata per landing and takes a few seconds each.

The two queues. Both take the same changes at the same times, with the same CI time.

• One at a time: each change lands one CI run after the later of its ready time and the previous landing.
• Sheltie: on every arrival and every finished build, the real scheduler plans the waiting changes. Changes that land now cost nothing extra: their own CI already passed and nothing that landed since touches them. A change that needs a rebuild, the merge engine or a combined build waits one CI run, together with its group. A change that conflicts with a group being built waits for that group.

The report shows the waiting time (total, median and 90th percentile), the CI runs each queue adds on top of the pull requests’ own CI, what Sheltie would have done with the changes, and the parallelism estimate from the same footprints.

What it does not claim

• Every CI run is assumed to pass, in both queues; red and flaky runs are not simulated.
• Changes without a known base are assumed to have been tested on the branch as it was when they landed. This matters: on six projects with merge queues, assuming that for every change cut Sheltie’s rebuilds from 36–348 to almost none (see Sheltie on real repositories). Fetch pull request heads for squash-merged repositories.
• Merge-engine resolutions are assumed to succeed. In practice some would go back to their authors.
• The comparison is with a one-at-a-time merge queue. If you merge by hand without a queue today, waiting time is not what you save; the shadow mode report on your real pull requests is the better measure.
• At least 10 landings with a footprint are needed.

Sheltie on real repositories

We measured what Sheltie would have done on 23 popular open-source repositories over 90 days (landings from 2026-07-21 to 2026-10-19). We compared it with what each project does today, not with a strawman. This page reports what we found, including where Sheltie does not help, where it would have let breakage through, the bugs in Sheltie the study exposed, and what we could not prove.

Every footprint here is the exact one Sheltie’s adapters compute for that commit. The queue results come from replaying each project’s real arrival times through Sheltie’s scheduler. They are simulations, not production runs.

The scripts, the bench and every number are in research/real-world.

Download this report as a PDF

Summary

The study also found problems in Sheltie, fixed before these results were computed (see What the study changed in Sheltie).

The projects and how they merge

We classified 26 projects from their GitHub rulesets, Actions events and pull request timelines. Three have no GitHub pull request merge flow to compare against:

• golang/go reviews on Gerrit;
• guava is exported from Google’s internal repository;
• spring-boot’s maintainers merge locally.

Whether the queue runs more than pull request CI decides how to read the results:

Queue projects: the real queue against Sheltie

How it was measured

• Ready time is when a pull request entered the merge process:
  • for GitHub merge queues, its last AddedToMergeQueue event;
  • for bors, the last “approved by” comment;
  • for traefik, when the merge label was added;
  • for Tide, when lgtm, approved, passing presubmits and the removal of blocking labels were all in place.
• Landed is the pull request’s mergedAt. (A merge-queue commit’s own timestamp is when its build started, not when it landed.)
• CI per build is the median wall time of a queue build:
  • for GitHub merge queues, merge_group runs grouped by commit, from first start to last finish;
  • for bors, its “Duration” line;
  • for gradle, whose CI runs on TeamCity, from pending to final of the gating status GitHub Merge Queue Check Pass on each landed commit.
• Sheltie replays the same entries at the same times through the scheduler Sheltie runs (plan_epoch), with the same CI time per run:
  • Each change counts as tested on main as it was when its own pull request CI last started.
  • If anything that landed after that writes what the change reads, Sheltie rebuilds it on the new head. If it writes what the change writes, the change goes to the merge engine.
  • Otherwise it lands at once.
• Like for like: the comparison uses only pull requests the real queue never ejected, because the replay assumes every build passes.
• Rebase merges: trino merges pull requests by rebasing several commits onto main. Each such pull request counts as one change, with one footprint from its first parent to its last commit.

Waiting

Waits are median / 90th percentile / total.

Read with care:

• rust-lang/rust (bors) runs 11 CI jobs on pull requests and 82 in bors. Landing at once would skip 71 job configurations, so its 88% is not a result Sheltie could deliver as is. Also, 222 of its 440 landings are rollups that bundle 2,151 pull requests; the replay sees a rollup as one change.
• traefik’s 37 changes in 90 days never overlapped in flight. The bot already lands them within minutes, so the saving is small in absolute terms (9 h 37 min in total).
• kubernetes is not in the table. Its waits are dominated by holds and policy (up to 35 days), and Prow’s CI is not visible on GitHub, so we have no CI time to replay with.
• bevy saves least because its changes overlap often. Of 592 changes, Sheltie would send 115 to the merge engine, rebuild 92 and build 57 together with a conflicting change.

CI runs

• Real queues build every entry, and rebuild entries behind every failure. Sheltie builds a change again only when something that landed after its CI overlaps it.
• For gradle the real count is a lower bound: one build per queue entry. Builds re-run after an ejection ahead of an entry are not visible.
• The last column is the fair cost for projects whose queue runs more than pull request CI: one full CI run per change plus Sheltie’s rebuilds. It is about the same as the real queue.

What Sheltie would have done with each change:

Would landing at once have let breakage through?

Queue failures, split by what Sheltie would have done with the same pull request:

Except at trino, queue failures hit pull requests that Sheltie would land at once as often as, or more often than, those it would rebuild. At trino and gradle, about a quarter of the pull requests Sheltie would land at once failed in the queue at least once. Footprint overlap is a weak predictor of which queue build fails.

Of the 223 pull requests that failed in the queue and that Sheltie would land at once, 165 went back into the queue with no new commits and merged. Those failures were most likely flaky or caused by another entry in the same build; we did not check each one. To find the rest, we took every pull request that failed in the queue, that Sheltie would land at once, and that got new commits afterwards. For each one we compared the pull request’s own diff at the time of the failure with its final diff, and read the failed jobs and follow-up commits:

• 18 only rebased or merged main; their diff did not change.
• 14 failed checks that run only in the queue, which Sheltie would need on the pull request:
  • 10 are repository-wide checks at gradle (distribution size, accepted API list, version lists, performance baselines). Any change can break these in combination with others, and no footprint captures them.
  • 4 at bevy, whose failing jobs run only in the queue.
• 14 are other gradle test or code fixes after a queue failure. Gradle’s pull request CI is a subset of its queue CI, so the failing test most likely did not run on the pull request, but we did not identify it (gradle’s queue CI is on TeamCity, outside the GitHub API).
• 2 changed trino’s CI workflows, which are only exercised in the queue.
• 1 is a semantic conflict with main (bevy: “Fix example merge conflict”). Footprints should have caught it and did not.
• 9 have a code change whose cause we could not determine: 5 at trino, 2 at rust-analyzer, 2 at bevy.

Projects without a merge queue

How it was measured

• CI is the build and test workflows that run both on pull requests and on pushes to main. Security audits, release, CodeQL, Copilot setup, zizmor, typos, CLA and labeller workflows are left out.
• Stale at merge: something else landed between the start of the pull request’s last CI and its merge.
• Sheltie would rebuild a pull request when one of those landings writes something it reads.
• Extra CI costs each such rebuild one full pull request CI run of wall time. That is an upper bound: Sheltie reruns only affected tests.
• Persistent red episode: two or more consecutive failed runs of a CI workflow on main. The candidates are the landings since the last green run. An episode counts as “with an overlapping landing” when a candidate was stale and something that landed in its gap overlaps it.
• Chance baseline: how many episodes would show an overlapping landing if overlap had nothing to do with breaking. Each candidate is taken to overlap at that project’s overall rate.

Results

n/m means not measurable:

• keycloak runs no CI on pushes to main, so red main is invisible;
• storybook’s main CI is not on GitHub Actions;
• only 38 of okhttp’s 147 pull requests had matching CI suites.

All persistent red episodes, by what their candidates looked like:

• stale base, overlapping landing: Sheltie rebuilds before landing: 83
• tested on the current main: not an interaction: 64
• stale base, no overlap: Sheltie lands it too: 27
• direct push or untested change in range: 2

• Stale merges are the norm, but they do not explain red main. Across 176 persistent episodes, an overlapping landing was in range 83 times, against 85.8 expected by chance. We found no evidence that Sheltie’s rebuilds would have prevented these breakages.
• 64 episodes started at changes tested on the current main. Flaky tests, infrastructure and environment changes break main without any interaction; vitest alone had 26.
• 27 episodes had stale changes with no overlap. If any of them was a real interaction, Sheltie’s footprints would have missed it too.
• Sheltie would cost CI here. It would rebuild 6–87% of pull requests. The largest shares come from coarse footprints: vscode’s root install script puts the whole repository in every read set, and keycloak’s Maven build plugins do the same for its test suite.

Adapter coverage

A footprint is leaky when the change reads something the adapter cannot model exactly:

• a build script;
• a proc macro;
• an install script;
• a build plugin;
• a file outside the module graph.

Its read set then widens to everything that leak could reach. Leaky does not mean serial, but it makes rebuilds more likely. This is the share of landings with a leaky footprint after the fixes below:

The causes that remain are mostly deliberate caution:

• Code that runs at build time: Cargo build scripts and proc macros, npm install scripts, Maven plugins that run code, Python setup.py extensions.
• Languages an adapter does not read:
  • ruff’s Python packaging;
  • pydantic’s Rust core;
  • prometheus’ web UI;
  • files under test/ read at run time.
• rust-lang/rust (95%): syn cannot parse the unstable syntax the compiler uses, so Sheltie claims nothing about those crates.
• bevy: a wasm32-only include_str! of a file that CI generates.

What the study changed in Sheltie

What we cannot prove

What was verified, and how

Reproduce it

research/real-world/README.md lists the steps: survey, collection, footprints, analyses and table generation. results/tables.md holds the tables above, generated from results/*.json.