pytest-geo vs Great Expectations for Spatial Validation

8 min read

Both a pytest suite built on Shapely and a Great Expectations suite can enforce the same spatial contract, and teams routinely agonize over which to standardize on. This comparison sits beneath choosing spatial testing tools and settles the decision on the axes that actually matter in a gate: who authors and reads the rules, whether the output must be a human-readable report, how the checks integrate with CI, and how each behaves as the rule set grows. The short version — imperative pytest for developer-owned geometry logic, declarative Great Expectations for broad column contracts published to non-developers — is only useful with the reasoning behind it, which is what this page provides.

The root difference: imperative vs declarative

pytest expresses validation as code: a function asserts a condition, and a failure is an exception with a traceback. Great Expectations expresses validation as data: a suite is a list of expectations over columns, and a failure is a structured result rendered into browsable data docs. That distinction cascades into everything else. Imperative checks are maximally flexible — any geometry predicate you can write in Shapely is one assertion away — but they are read by developers and their failures are tracebacks. Declarative expectations are more constrained but self-documenting, reviewable by a data owner who never opens the code, and they produce a report as a first-class artifact.

Two axes, not one

Framing this as “which is better” hides the trade-off, because the two tools are strong on different axes and neither dominates. Expressiveness — how arbitrary a rule can be — favours the imperative side without limit: anything Python can compute, a test can assert. Auditability — whether a non-programmer can read the rule set, review a change to it, and receive a report from it — favours the declarative side just as decisively.

Expressiveness against auditability A scatter chart with expressiveness increasing to the right and auditability increasing upward. A pytest assertion is plotted at high expressiveness and low auditability, annotated that any computable rule is expressible but the rule set is readable only by programmers and produces no rendered report. An expectation suite is plotted at moderate expressiveness and high auditability, annotated that the rules are declarative and reviewable and render into a report, but are limited to what the framework can express. A database constraint is plotted at low expressiveness and low auditability but marked as absolutely enforced. The upper-right quadrant, combining both strengths, is shaded and labelled as reachable only by running both tools together with shared thresholds. auditability expressiveness reachable only with both tools suite for contracts, pytest for the rest, one shared thresholds file pytest assertion any computable rule readable by programmers only expectation suite declarative, reviewable renders a report framework vocabulary only database constraint least expressive — and absolutely enforced Neither point dominates the other. Choosing one tool means choosing which axis to give up, which is why mature estates end up with both.

The axes also explain why the argument recurs. An engineer evaluating on expressiveness reaches an obvious conclusion; a data steward evaluating on auditability reaches the opposite one; and both are correct about the axis they weighted. Naming the axis before comparing turns a preference argument into a decision about what the organisation actually needs from this particular rule.

Comparison reference

Axis pytest + Shapely Great Expectations
Rule style Imperative assertions Declarative expectation suite
Best for Arbitrary geometry predicates Column contracts, domains, types
Audience Developers Data owners + developers
Failure output Traceback Data docs, structured result
Custom geometry rules Native (any Shapely call) Needs a custom expectation
CI integration Exit code, native Checkpoint returns success flag
Overhead at small scale Minimal Suite + context boilerplate

Where pytest with Shapely wins

When the rule is a geometry predicate — “no polygon self-intersects”, “every route connects end to end”, “this buffer contains that point within tolerance” — pytest is the natural home, because the check is one Shapely call and lives beside the code that produces the geometry. It also wins when the rules are few and developer-owned, where the ceremony of a Great Expectations context is pure overhead.

# pytest: an arbitrary geometry predicate, no framework overhead
import pytest, geopandas as gpd
from shapely import is_valid, hausdorff_distance

@pytest.mark.geometry
def test_routes_match_reference_within_tolerance():
    got = gpd.read_file("out/routes.gpkg").geometry
    ref = gpd.read_file("tests/fixtures/routes_ref.gpkg").geometry
    for g, r in zip(got, ref):
        assert is_valid(g)
        assert hausdorff_distance(g, r) <= 0.01     # metres, projected CRS

This is exactly the tolerance logic from spatial tolerance thresholds, expressed imperatively.

Where Great Expectations wins

When the contract spans many columns — required fields, dtypes, value domains, CRS, null policies — and must be reviewed or consumed by people who do not read Python, Great Expectations wins by producing a declarative, self-documenting suite and a data-docs report. It shines as the shared contract between a data producer and consumer, where the expectation suite is the interface. Geometry-specific rules require a custom expectation, covered under Great Expectations spatial expectations.

# Great Expectations: a declarative column contract, rendered to data docs
import great_expectations as gx

ctx = gx.get_context()
batch = ctx.data_sources.pandas_default.read_dataframe(gdf)   # gdf: GeoDataFrame
batch.expect_column_values_to_not_be_null("parcel_id")
batch.expect_column_values_to_be_in_set("land_use", ["residential", "commercial"])
batch.expect_column_values_to_be_between("elevation_m", -430, 8849)

Integrating each into a CI gate

Both integrate cleanly, but differently. pytest gates on its exit code — the model used throughout GitHub Actions spatial testing. Great Expectations runs a checkpoint and gates on the returned success flag, which you convert to an exit code.

# ge_gate.py — convert a checkpoint result into a gate exit code
import sys, great_expectations as gx

result = gx.get_context().run_checkpoint(checkpoint_name="parcels_contract")
sys.exit(0 if result.success else 1)

Many mature pipelines use both: pytest for geometry predicates and Great Expectations for the column contract, each owning the layer it fits — the anti-pattern is enforcing the same rule in both, which the choosing spatial testing tools framework warns against.

How the choice ages

The two tools diverge more as a rule set grows than they do at the start, and the divergence runs in both directions. A pytest suite gains rules cheaply for a long time and then hits a wall of a different kind: not performance, but legibility. Once several hundred assertions exist, nobody can answer “what do we actually require of this dataset” without reading code, and the answer becomes tribal knowledge. An expectation suite has the opposite curve — a higher fixed cost to stand up, then near-flat marginal cost, because each new rule is a line of JSON in a file whose whole purpose is to be read.

How each approach scales with rule count A chart of total effort against the number of rules. The pytest line begins at essentially zero setup cost and rises slowly, then bends upward steeply past a marked point where the rule set has become too large to summarise without reading source. The expectation suite line begins noticeably higher because of framework setup, then remains close to flat as rules are added. The two lines cross partway along, and the crossing point is annotated as roughly where a producing team or a data steward begins asking what the dataset is required to satisfy. A note observes that the crossing arrives earlier than most teams expect. total effort rules pytest expectation suite the crossing someone asks what the dataset must satisfy suite setup cost first test is free

The crossing arrives earlier than most teams expect, and it is triggered by an organisational event rather than by a rule count — the first time a producer, an auditor, or a new consumer asks what the pipeline requires. Teams that have only pytest answer that question by exporting a summary somebody hand-maintains, which is a suite with none of the guarantees.

One thresholds file, two enforcement tools A single versioned thresholds file sits at the centre. An arrow runs from it to an expectation suite, which is annotated as enforcing producer-facing data contracts and producing the rendered report sent to the producing team. A second arrow runs from the same file to a pytest suite, annotated as enforcing code-owned rules and gating the merge with rich assertion messages for engineers. Both tools are marked as containing no literal thresholds of their own. A footer states that this arrangement is what makes tightening a tolerance a single reviewed change rather than a search across two codebases. thresholds.yaml versioned · reviewed · single source expectation suite producer-facing contracts renders the report pytest suite code-owned rules gates the merge to the producing team counts, samples, trend to the engineer rule, feature id, value vs threshold Neither tool holds a literal. Tightening a tolerance becomes one reviewed change rather than a search across two codebases that may already disagree.

Failure modes and edge cases

  1. Duplicating a rule in both. The same CRS check in a pytest test and an expectation drifts over time; assign each rule to exactly one tool.
  2. Great Expectations for a single predicate. Standing up a context and suite to assert one geometry rule is overhead a pytest one-liner avoids.
  3. pytest as the non-developer contract. Burying a column contract in test code hides it from the data owner who needs to review it; that audience needs data docs.
  4. Custom expectation for a trivial geometry rule. Writing a custom expectation to check validity, when a pytest is_valid assertion suffices, adds maintenance for no gain.
  5. Version-API drift. Great Expectations changed its API across 0.15→0.18+; pin the version so the suite syntax matches the runtime, mirroring the runtime pinning in containerized runtimes.

A migration path that does not require a rewrite

Teams that start with pytest and later need the auditability rarely have to convert anything. The rules that belong in a suite are, almost by definition, the simple column-level ones — type, domain, nullability, range — and those are quick to express declaratively from scratch. The complex geometric assertions that would be painful to convert are exactly the ones that should stay in pytest anyway.

So the migration is additive: stand up a suite covering the contract a producer needs to see, delete the pytest duplicates of those specific rules, and leave everything else alone. It takes a day, it does not touch the interesting tests, and the end state is the hybrid rather than a replacement. Attempting the opposite — expressing every geometric predicate as a custom expectation — is the version of this that consumes a sprint and produces something harder to read than what it replaced.

Conclusion

The choice is not about capability — both can enforce most spatial rules — but about ownership and audience: pytest with Shapely for developer-owned geometry predicates, Great Expectations for broad column contracts that non-developers review as data docs. Assign each rule to the tool whose model fits it, never both, and gates stay maintainable as the rule set grows. For the full decision framework, return to choosing spatial testing tools.

A closing caution about the comparison itself: benchmark neither on a toy rule. Both tools handle a null check identically well, and any comparison built on one tells you nothing. Evaluate on the rule you actually find hardest — usually a geometric predicate with a negotiated tolerance and a producer who needs to see the result — because that is the rule whose home determines how the rest of the estate is organised.

Whichever tool a rule lands in, record the decision beside the rule rather than in a design document nobody reads at review time.