GitHub Actions Spatial Testing

13 min read

GitHub Actions is where most teams enforce their first spatial quality gate, because a workflow can be wired as a required status check that physically blocks a merge until the spatial assertions pass. This pattern sits beneath CI/CD spatial quality gates and covers the concrete mechanics: how to install the GDAL/PROJ stack quickly, how to run the fast pre-merge tier on every push, how to matrix across engine versions so an upgrade cannot silently change a result, and how to surface a failed geometry as a downloadable artifact instead of a wall of console text. The reason spatial CI on Actions needs its own treatment is that the default actions/setup-python step does not solve the hard part — the binary geometry libraries behind Shapely, GeoPandas and pyproj — and getting that wrong is what produces “passes locally, fails in CI” reports.

GitHub Actions spatial workflow from pull request to required status check A pull request triggers a workflow. A setup job restores a spatial-wheel cache. A matrix then fans out across two GDAL and PROJ pairs; each matrix job runs pytest-geo and uploads failed-geometry artifacts. A required status check aggregates the matrix result and gates the merge. Pull request Restore cache spatial wheels GDAL 3.8 · PROJ 9.3 pytest-geo GDAL 3.9 · PROJ 9.4 pytest-geo Artifacts failed geometry Required check

Every spatial workflow decomposes into the same four responsibilities: provision the binary stack reproducibly, run assertions as ordered tiers, matrix across the engine versions you support, and report failures as reviewable artifacts. The sections below take each in turn, with the two hardest — gating pull requests with pytest-geo and caching GDAL/PROJ wheels — expanded in their own worked guides.

Workflow Responsibility Reference

Responsibility Actions primitive What it prevents Typical budget
Provision stack Container image: or cached wheels Engine-version drift between runs 5–40 s
Run assertions pytest step, ordered markers Slow checks blocking fast feedback < 60 s (pre-merge)
Matrix engines strategy.matrix An upgrade silently changing a result ×N job time
Report failures actions/upload-artifact Non-reproducible console-only failures negligible
Enforce Branch protection required check Merging a spatial regression 0

Provisioning the Spatial Stack

The single most consequential choice in a spatial workflow is how the geometry libraries reach the runner. Three approaches dominate, and they trade reproducibility against speed. A container image: on the job gives exact GDAL, GEOS and PROJ versions with no install step, at the cost of pulling the image. Pinned wheels via pip install are fast when cached but only reproducible if every binary dependency is pinned. System packages via apt are fastest to write but least reproducible, because the distribution can bump GDAL under you. For a gate that must be deterministic, prefer a container or fully pinned wheels — the details of building that image live under containerized GIS test runtimes.

# .github/workflows/spatial.yml — container gives an exact, pinned stack
jobs:
  spatial:
    runs-on: ubuntu-24.04
    container:
      image: ghcr.io/acme/gis-test:gdal3.9.2-proj9.4.1   # pinned, not :latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -e '.[test]'      # app + pinned test extras
      - run: pytest -q -m "geometry and not slow"   # fast tier only

Ordered Assertion Tiers

Spatial assertions must run cheapest-first so a missing CRS never blocks behind a million-vertex Hausdorff comparison. Express the tiers with pytest markers and select them per gate: the pre-merge job runs not slow, the nightly job runs everything. The spatial assertion types taxonomy maps directly onto these markers — metadata and validity are fast, geometric tolerance checks are moderate, cross-engine joins are slow.

# conftest.py — register markers so unknown-marker warnings do not mask real ones
def pytest_configure(config):
    for m in ("geometry", "topology", "crs", "slow"):
        config.addinivalue_line("markers", f"{m}: spatial test tier")

# tests/test_geometry.py
import pytest, geopandas as gpd
from shapely import is_valid

@pytest.mark.geometry
def test_all_geometries_valid():
    gdf = gpd.read_file("tests/fixtures/parcels.gpkg")
    invalid = gdf[~gdf.geometry.map(is_valid)]
    assert invalid.empty, f"{len(invalid)} invalid geometries: {list(invalid.index)}"

Matrix Testing Across Engine Versions

Because GEOS and PROJ can change a result across builds, matrix the workflow over the engine versions your production runtimes actually use. A matrix that spans the version you run today and the version you plan to upgrade to turns an upgrade from a surprise into a green or red check before you cut over.

strategy:
  fail-fast: false          # let every cell report, do not cancel on first red
  matrix:
    include:
      - { image: "ghcr.io/acme/gis-test:gdal3.8.4-proj9.3.1" }
      - { image: "ghcr.io/acme/gis-test:gdal3.9.2-proj9.4.1" }
container:
  image: ${{ matrix.image }}

Set fail-fast: false so a failure in one engine pair does not cancel the others — when an upgrade regresses, you want to see exactly which version pair broke, not just that something did.

Reporting Failed Geometry as Artifacts

A spatial failure is far more debuggable as a file than as a stack trace. Serialize the offending features to GeoJSON on failure and upload them, so a reviewer can open the geometry in a viewer rather than reconstruct it from coordinates in a log — and so no raw coordinate needs to be pasted into a public log, which matters for the security boundaries around location PII.

# conftest.py — dump failing geometry for artifact upload
import json, os, pytest

@pytest.fixture
def geom_recorder():
    failures = []
    yield failures
    if failures:
        os.makedirs("artifacts", exist_ok=True)
        with open("artifacts/failed_geometry.geojson", "w") as fh:
            json.dump({"type": "FeatureCollection", "features": failures}, fh)

      - name: Upload failed geometry
        if: failure()
        uses: actions/upload-artifact@v4
        with: { name: failed-geometry, path: artifacts/ }
Job graph: one setup, parallel assertions, an advisory matrix A dependency graph of workflow jobs. A setup job resolves the pinned container and restores the dependency cache. Three assertion jobs — contract, geometry and parity — each depend on setup and run concurrently. A required status check job depends on all three and is the one branch protection enforces. Separately, a matrix job depends only on setup, runs the same suite against a candidate engine version, and reports on its own without feeding the required check, so an upstream release cannot block merges. A note observes that collapsing all of this into one sequential job makes the slowest check the latency of every result. setup container + cache contract geometry parity schema · CRS validity · topology formats round-trip required check branch protection enforces this matrix — candidate engine version reports independently · never required Collapsed into one sequential job, the slowest check becomes the latency of every result — and a parity failure hides behind a geometry failure.

Enforcing the Gate with Branch Protection

The workflow only becomes a gate when branch protection marks it required. Point the required check at the aggregating job (not each matrix cell) so the rule reads “spatial suite passed” regardless of how many engine pairs ran. This is the same enforcement model whether the assertions are written with pytest or delegated to Great Expectations — the choice between them is covered in choosing spatial testing tools.

What a Spatial Workflow Should Emit Besides a Verdict

A workflow that returns only pass or fail throws away most of what the run knew. Four artefacts, each cheap to produce, turn a red build from a starting point for investigation into a finished diagnosis — and they are the difference between a gate people trust and one they re-run.

The quarantine dataset. Every feature that failed, written as a GeoPackage with the rule name and the reason attached, uploaded as an artefact. An engineer downloads it, opens it in a desktop GIS, and sees the problem. Nothing else in a CI report substitutes for looking at the geometry.

The structured run summary. Engine versions, input content hash, source revision, per-rule counts. One machine-readable file that answers “was this the data or the environment” without a second run.

Inline annotations on the diff. Where a failure maps to a line — a changed threshold, an altered transformation — an annotation puts it in front of the reviewer rather than in a log they must open.

The metric series. Counts and measured drift, pushed to wherever the team watches trends, so a rule that is quietly degrading is visible before it crosses a threshold.

Five outputs, only one of which is the verdict A workflow run fans out into five outputs. The verdict, pass or fail, goes to the merge gate. A quarantine GeoPackage containing every failing feature with its rule and reason goes to an engineer, who opens it directly in a desktop GIS. A structured run summary carrying engine versions, the input content hash, the source revision and per-rule counts answers whether the cause was the data or the environment. Inline annotations attach failures to the relevant lines of the diff for the reviewer. A metric series of counts and measured drift is pushed to the dashboard where a slowly degrading rule becomes visible before it breaches any threshold. workflow run pinned container verdict — pass or fail → the merge gate quarantine.gpkg failing features + rule + reason → open it in a GIS run summary engine versions, input hash, counts → data or environment? inline annotations attached to the diff → the reviewer sees it metric series counts and measured drift → degradation, before the breach

The quarantine artefact is the one with the highest return and the lowest adoption. Writing failing features to a file costs a few lines in a fixture teardown, and it removes the single most common reason a spatial failure takes an afternoon: the engineer cannot see what failed. A rule name and a feature identifier tell you that something is wrong; the geometry tells you what, usually in seconds.

Keeping the Matrix Honest

A version matrix is the standard answer to engine drift, and it is easy to build one that costs a great deal and proves very little. Three properties separate a useful matrix from an expensive one.

It must vary only what it claims to vary. A matrix whose entries differ in GEOS version and base image and Python version cannot attribute a failure to any of them. One axis per matrix, and where several axes genuinely matter, run several matrices rather than a cross-product nobody can read.

It must not block the merge. The pinned combination gates; the matrix informs. Making a matrix entry required turns every upstream release into a merge outage, and the predictable response is to remove the entry rather than to investigate it.

Its failures need an owner and a cadence. A matrix that goes red and stays red for a quarter has become decoration. Reviewing it weekly, with someone accountable, is what keeps it a signal.

Matrix design Cost What a failure tells you
One axis: GEOS version, everything else pinned Low An engine change altered a result
One axis: PROJ data package Low A grid change moved coordinates
Cross-product of four axes High Something changed — unclear what
Matrix entries required for merge Low compute, high disruption Upstream released; your merges stopped

The most valuable single matrix in a spatial project is usually the narrowest one: the current pinned GEOS against the next release, on the same image, running the same suite. It costs one extra job, it answers the question that upgrade planning actually asks, and its failures are unambiguous.

Common Failure Modes and Gotchas

  1. setup-python without the binary stack. Installing Python does not install GDAL; the pip install shapely wheel bundles its own GEOS, but fiona/pyogrio need GDAL present. Use a container or a spatial-aware install action.
  2. :latest image tags. A floating tag makes the runtime non-deterministic; a rebuild of :latest can change GEOS and break a previously green gate. Always pin.
  3. fail-fast: true on a matrix. The default cancels sibling jobs on the first failure, hiding which engine pair actually regressed.
  4. Uncached wheels in the fast lane. A cold GDAL/PROJ install can dominate a sub-minute gate; without a cache the pre-merge tier stops being fast — see the caching guide.
  5. Raw coordinates in logs. Echoing failing geometry into the job log can leak location PII; write it to an artifact instead.
  6. Required check pointed at a matrix cell. If branch protection requires one specific cell, dropping that engine from the matrix silently disables the gate.

Frequently Asked Questions

Should the spatial stack come from a container or from a setup action?

A container, for anything whose result depends on the geometry engine. A setup action installs into the runner’s image, which means the surrounding system libraries are whatever the runner ships this week — and that is exactly the variable a spatial gate needs fixed. Running the job inside a pinned image makes the compiled layer identical to what a developer gets locally, which is the whole point.

How do we keep the workflow file readable as the suite grows?

Move logic out of the YAML. A workflow that calls one script per job stays short, and the script is testable locally in a way a workflow step never is. The rule of thumb: if a step contains a conditional, it belongs in a script. Workflows that accumulate shell logic become the least-reviewed, least-tested code in the repository, and spatial pipelines accumulate it faster than most because of environment handling.

Is it worth caching fixtures as well as dependencies?

Only if they are expensive to generate and their generator is hashable. A cache keyed on a hash of the generator source plus its configuration is safe; one keyed on a branch name or a date is how a stale fixture survives for weeks and quietly weakens every test that uses it. When in doubt, regenerate — deterministic generation is usually fast, and the correctness guarantee is worth more than the seconds.

What should happen when the workflow itself changes?

Run the full suite, including anything usually deferred. A change to the gate is the one case where the cheap-fast split does not apply, because the thing under test is the gate rather than the data. Scoping the workflow’s own changes to a reduced run is how a broken nightly job ships unnoticed.

How do we handle secrets for a private data source in a spatial job?

Keep them out of the pre-merge lane entirely, by making that lane run against generated fixtures. Pull requests from forks cannot access secrets in most configurations, and a gate that only works for internal branches is not a gate. Where a job genuinely needs credentials — a nightly run against a real store — put it on a separate workflow with its own trigger, so the fast lane never depends on them.

Should annotations be produced for every failure?

For failures that map to a line, yes. For a data failure with no diff location — a fixture that drifted, an upstream dataset that changed — an annotation attached to an arbitrary line is worse than none, because it implies a cause that is not there. Those belong in the job summary and the quarantine artefact, where their actual context lives.

Keeping the fast lane fast as the suite grows

The pre-merge lane degrades gradually and nobody notices until it is too slow to be useful. Two habits keep it honest. First, record the lane’s duration as a metric and alert on the trend rather than on a threshold — a lane that has grown from ninety seconds to four minutes over a quarter is a problem long before it crosses five. Second, require that any new test declares which lane it belongs in, and make the fast lane’s membership a deliberate decision rather than the default.

The alternative, discovered by most teams eventually, is a periodic emergency in which someone spends a week splitting a suite that grew organically. Measuring from the start turns that into a series of small decisions made while the context is still fresh.

Reusable workflows across repositories

Once more than one repository gates on spatial rules, the workflows drift apart within a quarter. A reusable workflow, called from each repository with a small set of inputs, keeps the gate’s shape identical everywhere while letting each project name its own paths and markers.

What belongs in the reusable definition is the structure: the container, the job graph, the artefact handling, the summary format. What belongs in the caller is everything project-specific. The failure mode to avoid is a reusable workflow with a dozen inputs, which is a copy of every caller’s specifics wearing a shared name — at that point the sharing has cost more than it saved.

Conclusion

A GitHub Actions spatial gate is four disciplined choices: a pinned binary stack, cost-ordered assertion tiers, a matrix over the engine versions you support, and failures reported as artifacts — all enforced by a required status check. Built this way, the workflow blocks a spatial regression at the pull request rather than discovering it in production, and it does so reproducibly. For the platform-neutral gate architecture these workflows implement, return to CI/CD spatial quality gates.

The workflow is infrastructure, and it deserves the same review standard as the code it protects: small jobs, named clearly, doing one thing each, with logic in scripts that can be run by hand. A gate nobody can read is a gate nobody will fix.

The smallest useful spatial workflow

It is worth knowing what the minimum looks like, because teams frequently postpone the gate while designing something comprehensive. A workflow that runs on pull requests, inside a pinned image, executing the contract checks alone — schema present, CRS declared and correct, geometry valid — and marked as a required check, is perhaps twenty lines and catches a substantial share of real regressions.

Everything else in this section is an improvement on that base rather than a prerequisite for it. Matrix runs, quarantine artefacts, caching and annotations each add value, and none of them is worth delaying the first enforced check for. The gate that exists imperfectly today prevents more regressions than the complete one that ships next quarter.