Gating Pull Requests with pytest-geo in GitHub Actions

9 min read

A spatial test suite only prevents regressions if a failing assertion physically stops the merge. This guide sits beneath GitHub Actions spatial testing and shows exactly how to turn a pytest run of your spatial checks into a required status check on GitHub, so a pull request that breaks topology, CRS or schema validation cannot be merged until it is fixed. The parts that trip teams up are not the assertions themselves — those come from spatial assertion types — but the wiring: making the check appear on the pull request, ensuring a failing test produces a non-zero exit that the platform sees, and configuring branch protection to require the right job.

Why a green suite still lets regressions through

Three configuration gaps let a spatial regression merge even when tests exist. First, the workflow never runs on the pull request because it is triggered only on push to a branch, so GitHub has no check to require. Second, a test failure does not propagate: a continue-on-error step, a piped command that swallows the exit code, or a shell that returns the last command’s status masks the failure. Third — the most common — the suite runs and reports, but branch protection was never told to require it, so the red check is advisory. Closing all three is what makes the gate real.

The four ways a “passing” gate lets a regression through

A green check is not the same as an enforced gate, and the gap between them has exactly four common causes. Each is invisible from the pull request, which is why they persist for months.

Four ways a green check gates nothing Four failure modes with detections. First, the workflow is not configured as a required status check, so a merge is allowed whatever it reports; detected by reading branch protection rather than by trusting the workflow. Second, the exit code is lost — swallowed by a shell pipeline or by a continue-on-error setting — so a failing suite reports success; detected by deliberately breaking a test and confirming the check goes red. Third, path filters skip the job for the changed files, and a skipped job is reported as green; detected by checking whether the job ran at all rather than whether it passed. Fourth, the suite collects zero tests because a marker expression no longer matches anything, and pytest exits successfully; detected by asserting a minimum collected count. FAILURE MODE WHY IT LOOKS GREEN HOW TO DETECT IT Not a required check branch protection never mentions it merge proceeds regardless read the protection rule, not the YAML Exit code swallowed a pipe, or continue-on-error failing suite reports success break a test on purpose, expect red Path filter skipped it changed files not matched skipped reports as green assert the job ran, not that it passed Zero tests collected a marker was renamed an empty run exits 0 assert a minimum collected count

The fourth is the most insidious because it arrives through an ordinary refactor. Renaming a marker, moving a directory, or tightening a selection expression can silently reduce the collected set to nothing, and a run that collects nothing exits successfully. Asserting a floor on the collected count — a single line in configuration — converts that from an invisible loss of coverage into an immediate, obvious failure.

Configuration reference

Setting Value that gates Value that silently fails open
Trigger on: pull_request on: push to a feature branch only
Step exit run: pytest (bare) `run: pytest
Aggregation Required check points at one summary job Required check points at a matrix cell that may be dropped
Branch protection “Require status checks to pass” enabled Check exists but not marked required
Annotations pytest + a reporter that emits ::error:: Plain output, no inline diff annotations

Step-by-step implementation

The workflow below targets pytest 7+, Shapely 2.x and GeoPandas 0.14+, and produces a single required check named spatial-gate.

Step 1 — Trigger on the pull request

# .github/workflows/spatial-gate.yml
name: spatial-gate
on:
  pull_request:
    branches: [main]        # runs on every PR targeting main

Step 2 — Run the suite so failures propagate

Run pytest as a bare step. Do not append || true, and if you pipe output, set shell: bash with pipefail so the pytest exit code — not tee’s — decides the step.

jobs:
  spatial-gate:
    runs-on: ubuntu-24.04
    container: { image: "ghcr.io/acme/gis-test:gdal3.9.2-proj9.4.1" }
    steps:
      - uses: actions/checkout@v4
      - run: pip install -e '.[test]'
      - name: Run spatial assertions
        run: pytest -q -m "not slow" --junitxml=report.xml

Step 3 — Emit inline annotations on the diff

A small conftest.py hook turns each failure into a GitHub ::error:: annotation so the reviewer sees it on the changed line, not only in the log.

# conftest.py — annotate failures inline on the PR
import os

def pytest_runtest_logreport(report):
    if report.failed and os.environ.get("GITHUB_ACTIONS"):
        loc = report.location  # (path, lineno, domain)
        print(f"::error file={loc[0]},line={(loc[1] or 0) + 1}::{report.nodeid} failed")

Step 4 — Make branch protection require the job

In the repository settings, under branch protection for main, enable “Require status checks to pass before merging” and select spatial-gate. Point it at this summary job, never at an individual matrix cell — if you later drop that engine from a matrix, a cell-level required check silently disappears and the gate opens.

Testing the gate itself

Every other check in the repository is verified by the gate; the gate is verified by nobody unless someone arranges it. Two lightweight practices close that hole, and both take minutes.

The first is a negative-control test: a test in the suite that is designed to fail, disabled by default, and enabled by an environment variable. A scheduled job runs the workflow with it enabled and asserts that the check goes red. If it does not, something between the assertion and the merge button is broken, and you have learned it on a quiet Tuesday rather than after a regression ships.

The second is an audit of the protection rule rather than of the workflow. What blocks a merge is the branch protection configuration, not the YAML, and the two drift apart whenever a job is renamed. A short scheduled script that compares the required check names against the jobs the workflow actually produces catches the rename the day it happens.

Two scheduled controls that keep the gate honest Two independent scheduled jobs. The first enables a negative-control test that is designed to fail, runs the gate workflow, and asserts that the required status check reports failure; if it reports success, some link between the assertion and the merge button is broken. The second reads the job names the workflow produces and compares them against the names branch protection lists as required, reporting any mismatch, which is how a renamed job that silently stopped gating is caught. Both are marked as running on a schedule rather than on the merge path, so their own failures inform rather than disrupt. scheduled control 1 enable the failing test off by default, on by env var assert the required check goes red green here means the path to the merge button is broken scheduled control 2 list the workflow’s job names what the run actually produces compare against the required check names a rename that stopped gating shows up the same day Both run on a schedule, off the merge path, so their own failures are information rather than disruption. Every other check is verified by the gate. Only these verify the gate.

There is a third practice worth adopting when the stakes justify it: require the gate on the default branch as well, so a direct push cannot bypass what a pull request could not. Teams frequently protect the merge path carefully and leave the administrative override entirely unmonitored, which means the one route most likely to be used under pressure is also the least observed.

Three routes into the default branch Three paths converge on the default branch. The first is a pull request, which passes through the required status check and is therefore gated. The second is an administrative override of a failing check, which bypasses the gate and is only visible if overrides are separately logged and reviewed. The third is a direct push by a user with elevated permissions, which bypasses the check entirely unless the branch protection rule explicitly includes administrators. A closing note observes that both ungated routes are the ones taken under time pressure, which is precisely when the gate would have been most valuable. pull request admin override direct push required check — gated bypasses the check visible only if overrides are logged bypasses the check unless protection includes admins default branch Both ungated routes are the ones taken under time pressure — which is exactly when the gate would have been most valuable.

Verification pattern

Prove the gate blocks by opening a throwaway pull request that introduces an invalid geometry and confirming the check goes red and the merge button is disabled. A local dry run of the exact command the workflow uses catches most issues before you push:

pytest -q -m "not slow" --junitxml=report.xml; echo "exit=$?"

An exit=1 here is what GitHub converts into a failing required check. If the suite exits 0 on known-bad data, the propagation is broken — fix Step 2 before trusting the gate.

Failure modes and edge cases

  1. Fork pull requests lack secrets. A pull_request from a fork runs without repository secrets; if your suite needs a database DSN, use pull_request_target cautiously or provide a service container, or the gate errors on forks rather than gating.
  2. Skipped tests read as passing. If fixtures are missing on the runner, pytest may skip the spatial tests and exit 0; assert collection with --strict-markers and a minimum test count so an empty run cannot pass.
  3. Path filters hide the check. Adding paths: to the trigger so the workflow skips non-spatial changes means the required check never reports on those PRs — GitHub then blocks the merge waiting for a check that will never run. Use a passing “no-op” job for filtered paths.
  4. Matrix cell as the required check. Requiring spatial-gate (gdal3.8) instead of the summary job couples the gate to one engine version.
  5. continue-on-error upstream. A lenient earlier step that installs the stack with continue-on-error: true can let the suite run against a broken environment and pass trivially.

What a good failure comment contains

When the gate fails, the comment or annotation it leaves is the only thing most reviewers will read. Four elements make it actionable: the rule that failed, the feature identifier or file where it failed, the measured value against the threshold, and a one-line reproduction command. Anything beyond that is noise; anything less sends the reader to the raw log.

The reproduction command is the element most often omitted and the one that saves the most time. A line the engineer can paste to run exactly the failing check locally, against the same fixture, converts a CI failure into a local debugging session in seconds rather than after a round of guessing at markers and paths.

Conclusion

A pull request gate is three correct choices — trigger on pull_request, let the pytest exit code propagate, and mark the summary job required in branch protection — plus inline annotations so failures land on the diff. With those in place, a spatial regression cannot reach main. For the broader workflow shape this fits into, return to GitHub Actions spatial testing.

Where a matrix feeds the gate, point the required check at an aggregating job rather than at individual cells, so a dropped or renamed cell cannot silently remove itself from the requirement.

A gate that has never been observed to fail has not been shown to work.

The wiring described here is unglamorous and it is where the value sits. Assertions are the easy part of a spatial gate; making a failing assertion reliably stop a merge, with a message someone can act on, is what turns a suite into a control.