Great Expectations Spatial Expectations

4 min read

Great Expectations gives spatial teams a declarative way to express a data contract — a suite of expectations over columns that validates, documents and reports in one artifact. This body of work sits within test data generation and mocking strategies and covers how to bend a tabular validation framework to geospatial data: expressing CRS, schema and domain rules as built-in expectations, writing custom expectations for the genuinely spatial checks (validity, topology, coordinate bounds) that no built-in covers, and wiring a checkpoint into CI so the suite becomes a gate. The reason this needs its own treatment is that Great Expectations has no native concept of a geometry column; making it validate spatial data well is a matter of knowing which rules map onto built-ins and which require the custom-expectation path detailed in writing custom geometry expectations.

Composing a spatial expectation suite from built-in and custom expectations A GeoDataFrame input splits into two paths. Non-geometry columns map to built-in expectations for schema, domain and CRS. The geometry column maps to custom expectations for validity, bounds and topology. Both paths feed a single expectation suite; a checkpoint runs it, producing a pass/fail result and data docs. GeoDataFrame the batch Built-in expectations schema · domain · CRS Custom expectations validity · bounds · topology Expectation suite one contract Checkpoint gate + data docs

The split in that diagram is the whole design: everything non-geometric is a built-in expectation, and everything genuinely spatial is a custom one. The sections below cover both halves, plus the checkpoint that turns the suite into a gate. Whether to reach for Great Expectations at all — versus plain pytest — is the subject of the pytest-geo vs Great Expectations comparison.

Expectation Coverage Reference

Rule Expectation Built-in or custom
Required field present expect_column_to_exist Built-in
Value domain expect_column_values_to_be_in_set Built-in
Numeric range expect_column_values_to_be_between Built-in
Non-null geometry expect_column_values_to_not_be_null Built-in (on WKT column)
Declared CRS matches custom expect_crs_to_be Custom
Geometry validity custom expect_geometry_to_be_valid Custom
Within bounds custom expect_geometry_within_bbox Custom
No self-overlap custom, batch-level Custom

Built-In Expectations on the Attribute Columns

The attribute layer of a spatial contract — required fields, dtypes, value domains, numeric ranges — maps directly onto built-in expectations, because those columns are ordinary tabular data. Represent geometry as a WKT or WKB column and even a non-null geometry check becomes a built-in. This is the same attribute-contract discipline as attribute and metadata checks, expressed declaratively.

import great_expectations as gx

ctx = gx.get_context()
batch = ctx.data_sources.pandas_default.read_dataframe(df)   # df has a wkt column
batch.expect_column_to_exist("parcel_id")
batch.expect_column_values_to_not_be_null("wkt")
batch.expect_column_values_to_be_in_set("land_use", ["residential", "commercial"])
batch.expect_column_values_to_be_between("elevation_m", -430, 8849)

Custom Expectations for the Geometry Column

The genuinely spatial rules — is the geometry valid, does it fall inside an expected bounding box, is its CRS the declared one — have no built-in, so they require a custom column-map expectation that parses the WKT and applies a Shapely predicate per row. The full mechanics of subclassing and registering one are in writing custom geometry expectations; the shape is a per-value function returning a boolean.

from shapely import from_wkt, is_valid

def _is_valid_wkt(value: str) -> bool:
    try:
        return bool(is_valid(from_wkt(value)))
    except Exception:
        return False       # unparseable WKT fails the expectation, never raises

Checkpoints as CI Gates

A suite becomes a gate when a checkpoint runs it and returns a success flag you convert to an exit code — the integration model from choosing spatial testing tools. The same run publishes data docs, so a failure is both a red gate and a browsable report a data owner can read.

import sys, great_expectations as gx

result = gx.get_context().run_checkpoint(checkpoint_name="parcels_spatial")
sys.exit(0 if result.success else 1)      # non-zero blocks the merge

Pipeline Integration

Pin the Great Expectations version alongside the spatial stack — its API shifted across 0.15→0.18+, so an unpinned upgrade can break suite syntax exactly like an unpinned GEOS breaks a predicate, the concern the containerized GIS test runtimes work formalizes. Run the checkpoint in the same job that runs the rest of the spatial gate, and treat the generated data docs as an artifact so failures are reviewable without re-running.

Common Failure Modes and Gotchas

  1. Expecting a geometry built-in to exist. There is no expect_column_values_to_be_valid_geometry; validity, bounds and topology all require custom expectations.
  2. Custom expectation that raises on bad input. A per-value function must return False on unparseable WKT, not raise, or one malformed row aborts the whole suite.
  3. Unpinned framework version. A minor Great Expectations upgrade can change the suite API; pin it so the syntax matches the runtime.
  4. CRS checked as a string. Comparing crs as text ("EPSG:3857" vs "epsg:3857") gives false failures; normalize to the authority code before asserting.
  5. Batch-level topology as a column-map. No-overlap is a relationship across rows, not a per-value check; implement it as a batch-level expectation, not a column-map one.

Conclusion

Making Great Expectations validate spatial data is a clean division of labour: built-in expectations for the attribute columns, custom column-map and batch-level expectations for the geometry, and a checkpoint that turns the whole suite into a CI gate with published data docs. Built this way, a spatial contract is enforced, documented and reviewable in one artifact. For the data-generation context this sits in, return to test data generation and mocking strategies.