Great Expectations Spatial Expectations

13 min read

Great Expectations gives spatial teams a declarative way to express a data contract — a suite of expectations over columns that validates, documents and reports in one artifact. This body of work sits within test data generation and mocking strategies and covers how to bend a tabular validation framework to geospatial data: expressing CRS, schema and domain rules as built-in expectations, writing custom expectations for the genuinely spatial checks (validity, topology, coordinate bounds) that no built-in covers, and wiring a checkpoint into CI so the suite becomes a gate. The reason this needs its own treatment is that Great Expectations has no native concept of a geometry column; making it validate spatial data well is a matter of knowing which rules map onto built-ins and which require the custom-expectation path detailed in writing custom geometry expectations.

Composing a spatial expectation suite from built-in and custom expectations A GeoDataFrame input splits into two paths. Non-geometry columns map to built-in expectations for schema, domain and CRS. The geometry column maps to custom expectations for validity, bounds and topology. Both paths feed a single expectation suite; a checkpoint runs it, producing a pass/fail result and data docs. GeoDataFrame the batch Built-in expectations schema · domain · CRS Custom expectations validity · bounds · topology Expectation suite one contract Checkpoint gate + data docs

The split in that diagram is the whole design: everything non-geometric is a built-in expectation, and everything genuinely spatial is a custom one. The sections below cover both halves, plus the checkpoint that turns the suite into a gate. Whether to reach for Great Expectations at all — versus plain pytest — is the subject of the pytest-geo vs Great Expectations comparison.

Expectation Coverage Reference

Rule Expectation Built-in or custom
Required field present expect_column_to_exist Built-in
Value domain expect_column_values_to_be_in_set Built-in
Numeric range expect_column_values_to_be_between Built-in
Non-null geometry expect_column_values_to_not_be_null Built-in (on WKT column)
Declared CRS matches custom expect_crs_to_be Custom
Geometry validity custom expect_geometry_to_be_valid Custom
Within bounds custom expect_geometry_within_bbox Custom
No self-overlap custom, batch-level Custom

Built-In Expectations on the Attribute Columns

The attribute layer of a spatial contract — required fields, dtypes, value domains, numeric ranges — maps directly onto built-in expectations, because those columns are ordinary tabular data. Represent geometry as a WKT or WKB column and even a non-null geometry check becomes a built-in. This is the same attribute-contract discipline as attribute and metadata checks, expressed declaratively.

import great_expectations as gx

ctx = gx.get_context()
batch = ctx.data_sources.pandas_default.read_dataframe(df)   # df has a wkt column
batch.expect_column_to_exist("parcel_id")
batch.expect_column_values_to_not_be_null("wkt")
batch.expect_column_values_to_be_in_set("land_use", ["residential", "commercial"])
batch.expect_column_values_to_be_between("elevation_m", -430, 8849)

Custom Expectations for the Geometry Column

The genuinely spatial rules — is the geometry valid, does it fall inside an expected bounding box, is its CRS the declared one — have no built-in, so they require a custom column-map expectation that parses the WKT and applies a Shapely predicate per row. The full mechanics of subclassing and registering one are in writing custom geometry expectations; the shape is a per-value function returning a boolean.

from shapely import from_wkt, is_valid

def _is_valid_wkt(value: str) -> bool:
    try:
        return bool(is_valid(from_wkt(value)))
    except Exception:
        return False       # unparseable WKT fails the expectation, never raises

Deciding What Belongs in a Suite at All

Great Expectations is a data-quality tool, and a spatial pipeline has quality concerns that belong in three different places. Putting all of them in a suite produces a slow, brittle artefact that duplicates the test suite; putting none of them there wastes the one tool that produces a report a data steward can read.

The dividing line is who acts on the failure. An expectation should exist when the answer is “the team that produces the data” and when the finding is about this batch rather than about the code. A pytest assertion should exist when the answer is “the engineer who changed the code”. A constraint should exist when the answer is “nobody — it must simply be impossible”.

Concern Belongs in Because
Attribute domain, nullability, ranges Expectation suite Producer acts; the result is a per-batch report
Feature counts within expected bounds Expectation suite Detects a truncated or doubled load
Geometry validity across a batch Expectation suite via a custom expectation Producer acts; the fraction matters
Tolerance arithmetic in your transform pytest Engineer acts; nothing to do with the batch
Predicate selection and CRS handling pytest A code property, constant across batches
Uniqueness of a primary key Database constraint Must be impossible, not merely detected
No two parcels overlap Database constraint or topology check A set property, not a column property
Choosing between a suite, a test and a constraint A single question — who acts on the failure — branches three ways. The producing team acting on a per-batch finding leads to an expectation suite, whose output is a validation report attached to the batch. The engineer who changed the code acting on a property that is constant across batches leads to a pytest assertion in the pre-merge gate. Nobody acting, because the state must be impossible, leads to a database constraint that rejects the write outright. Two warnings are recorded: a code property placed in a suite runs needlessly on every batch forever, and a batch property placed in pytest becomes dependent on whichever fixture happens to be loaded. Who acts on the failure? The producing team finding is about this batch → expectation suite The engineer who changed code property is constant across batches → pytest, in the merge gate Nobody — make it impossible the state must never exist → database constraint Misplacing it in either direction has a cost: a code property in a suite runs on every batch forever for no new information, and a batch property in pytest becomes a statement about whichever fixture happened to be loaded that day.

The middle column is where most of the confusion sits, and one test settles it: would this assertion give a different answer on a different batch of data? If no, it is about the code and belongs in pytest. If yes, it is about the data and belongs in a suite. Geometry validity is the interesting case, because it is genuinely both — the rule is a code property, but the fraction of features that violate it is a batch property, which is why validity usually appears as a custom expectation reporting an unexpected fraction rather than as a pass/fail assertion.

How a Suite Earns Its Keep Over Time

The value of a suite is not the first run; it is the twentieth, when the validation results form a series. Three uses emerge that no assertion-based approach provides.

Trend detection. The unexpected fraction for a rule is a time series. A rule at a steady 0.4 per cent that jumps to 3 per cent has detected an upstream change even though the threshold — set at 5 per cent — was never breached. Alerting on the change rather than only on the threshold catches problems days earlier.

Negotiation with producers. A report showing exactly which rules a batch violated, with counts and samples, is a document two teams can work from. “Your data is bad” starts an argument; “4.1 per cent of features have a null surface_type, against a contract of 1 per cent, sample attached” starts a fix.

Onboarding a new source. Pointing an existing suite at a new dataset produces an immediate, structured account of how that dataset differs from what the pipeline expects. That is a day-one gap analysis for free, and it is far more reliable than reading a specification the producer wrote about their own data.

None of these survive if the validation results are discarded after each run. Persisting them — even as JSON in object storage keyed by batch and suite version — is the cheap step that converts a checking tool into a monitoring one.

Checkpoints as CI Gates

A suite becomes a gate when a checkpoint runs it and returns a success flag you convert to an exit code — the integration model from choosing spatial testing tools. The same run publishes data docs, so a failure is both a red gate and a browsable report a data owner can read.

import sys, great_expectations as gx

result = gx.get_context().run_checkpoint(checkpoint_name="parcels_spatial")
sys.exit(0 if result.success else 1)      # non-zero blocks the merge

Pipeline Integration

Pin the Great Expectations version alongside the spatial stack — its API shifted across 0.15→0.18+, so an unpinned upgrade can break suite syntax exactly like an unpinned GEOS breaks a predicate, the concern the containerized GIS test runtimes work formalizes. Run the checkpoint in the same job that runs the rest of the spatial gate, and treat the generated data docs as an artifact so failures are reviewable without re-running.

Keeping the Gate Fast Enough to Keep

A checkpoint that takes twenty minutes will be moved to a nightly schedule, and a nightly gate does not block anything. Three techniques keep the pre-merge checkpoint in the range where it stays a gate.

Split by cost, not by subject. Column-level rules over a batch are cheap and run on everything. Geometry-level rules are expensive and run on a sample pre-merge and the full batch nightly. Splitting the suite in two along that line, rather than by “attributes” versus “geometry”, is what keeps the fast one fast as the schema grows.

Push the predicate to the store where possible. A suite executing against a database backend can express most column rules as SQL that the engine evaluates over an index, rather than pulling the batch into memory first. The difference on a multi-million-row table is minutes against seconds, and the rules read identically.

Sample deterministically, and say so. When a rule runs against a subset, the sample must be reproducible from the batch identifier, and the report must record that it was a sample and how large. An undocumented sample produces the worst of both worlds: a result nobody trusts and a runtime nobody understands.

One suite, two checkpoints, split by cost A single expectation suite feeds two checkpoints. The pre-merge checkpoint runs column rules across the entire batch with the predicates pushed down to the store, plus geometry rules across a deterministic sample, completing in seconds and therefore able to block a merge. The nightly checkpoint runs every rule against the full batch, completing in minutes, and its results feed the trend series used for alerting on change. An arrow from the nightly checkpoint back to the pre-merge one marks a rule graduating once it has been made cheap enough to run on every change. A footer states that the split is by evaluation cost rather than by subject matter. One suite versioned JSON Pre-merge checkpoint · seconds column rules · full batch · pushed down to the store geometry rules · deterministic sample, size recorded Nightly checkpoint · minutes every rule · full batch · no sampling results persisted → the trend series can block a merge fast enough to stay a gate alerts on change not only on the threshold a rule graduates once it is cheap enough Split by cost, never by subject.

Frequently Asked Questions

Do we need a data context, or can suites be built in code?

Both work, and the choice is about who edits them. A file-based context gives you suites as reviewable JSON, generated documentation, and a place to persist validation results — which is most of the value for a team with a data steward. Building suites in code is simpler for a pipeline whose rules are derived from a schema anyway, and avoids a directory structure nobody else touches. What you lose is the artefact a non-programmer can read, so decide on that basis rather than on ergonomics.

How do expectations coexist with the pytest suite without duplicating rules?

By keeping them on opposite sides of the batch-versus-code line and sharing the values rather than the rules. Thresholds live in one versioned configuration file that both read, so a tolerance is defined once. The rules themselves should not overlap: if a rule appears in both places, one of the two is testing the wrong thing, and it is almost always the pytest copy asserting on a fixture that happens to satisfy a data contract.

What is the right batch size for validation?

The unit your pipeline actually produces — one delivery, one partition, one day. Validating a smaller slice produces fractions that swing wildly and trend badly; validating an accumulation of several deliveries hides which one was bad. Matching the batch to the delivery also makes the result attributable, which is the property that lets you send the report to a producer.

Should a failed checkpoint block the pipeline or quarantine the batch?

Quarantine, in almost every case. Blocking leaves the pipeline stopped and the data unavailable, which turns a quality problem into an availability incident; quarantining routes the batch aside, keeps yesterday’s good data serving, and gives a human the artefact to look at. Reserve outright blocking for the small set of failures where serving stale data is safer than serving nothing.

How much of the suite should be generated from the schema?

As much as is mechanical — type, nullability, column presence, and any enumeration the schema already declares. Regenerating those on every schema change keeps them accurate at no cost. Everything that encodes judgement, particularly tolerated fractions and value ranges derived from domain knowledge, stays hand-written, and the two should live in separate files so a regeneration can never overwrite a decision.

Reading a Validation Result Well

The structured result is the artefact this whole approach exists to produce, and getting value from it is a skill worth stating explicitly. Four fields carry almost all of the signal.

The unexpected fraction, not the boolean. A rule that failed at 0.02 per cent and a rule that failed at 40 per cent are different problems with different owners, and the pass/fail flag flattens them into the same thing.

The unexpected sample. A handful of actual offending values usually identifies the cause outright — a single upstream system emitting a legacy code, a batch where one municipality used a different convention. Reading the sample is nearly always faster than reasoning about the rule.

The element count. A rule evaluated against far fewer rows than expected means the batch was truncated, or a filter upstream removed more than intended, and that is often a bigger problem than the rule’s own result.

The suite and expectation version. Without it, comparing today’s result against last month’s compares two different questions, and the trend that looked like improving data quality may just be a loosened threshold.

A report that surfaces those four for every failing rule turns validation output into something a producer can act on without a conversation. One that shows only a list of red rule names produces a meeting instead.

Common Failure Modes and Gotchas

  1. Expecting a geometry built-in to exist. There is no expect_column_values_to_be_valid_geometry; validity, bounds and topology all require custom expectations.
  2. Custom expectation that raises on bad input. A per-value function must return False on unparseable WKT, not raise, or one malformed row aborts the whole suite.
  3. Unpinned framework version. A minor Great Expectations upgrade can change the suite API; pin it so the syntax matches the runtime.
  4. CRS checked as a string. Comparing crs as text ("EPSG:3857" vs "epsg:3857") gives false failures; normalize to the authority code before asserting.
  5. Batch-level topology as a column-map. No-overlap is a relationship across rows, not a per-value check; implement it as a batch-level expectation, not a column-map one.

Where the suite lives in the repository

Treat the suite directory as source, not as generated output. It belongs beside the pipeline code, in review, with the same branch protection — because a change to a threshold is a change to what the system accepts, and reviewing it is the only moment anyone will notice a loosened rule. Storing suites in an object bucket that the pipeline reads at runtime removes that review step entirely, and the predictable result is a set of thresholds nobody remembers agreeing to.

The validation results, by contrast, are genuinely generated output and belong in storage rather than in the repository. Keeping the two apart — definitions in git, results in a bucket keyed by batch — is what makes the trend queryable without turning the repository into a data store.

A related discipline is to keep the number of suites small. One suite per dataset, versioned, is manageable; one per pipeline stage multiplies quickly and leaves nobody able to say which rules a given batch was actually checked against. Where different stages genuinely need different rules, express that as separate checkpoints referencing the same suite rather than as separate suites.

Conclusion

Making Great Expectations validate spatial data is a clean division of labour: built-in expectations for the attribute columns, custom column-map and batch-level expectations for the geometry, and a checkpoint that turns the whole suite into a CI gate with published data docs. Built this way, a spatial contract is enforced, documented and reviewable in one artifact. For the data-generation context this sits in, return to test data generation and mocking strategies.

One organisational note worth stating: give each suite a named owner, recorded in the suite metadata rather than in a wiki. A suite without an owner accumulates rules nobody can explain, and the first time one fires unexpectedly the whole file gets disabled because there is no one to ask whether the rule was right. Naming an owner costs a line and is the difference between a suite that is maintained and one that is merely present.