Writing Custom Geometry Expectations in Great Expectations

3 min read

Great Expectations ships no geometry-aware expectation, so validating that a WKT column contains valid, in-bounds geometry means writing a custom one. This guide sits beneath Great Expectations spatial expectations and walks the full path for a column-map expectation — subclassing the base, defining a Shapely-backed metric, registering it, and using it in a suite — using geometry validity as the worked example. The mechanics generalize to any per-row spatial predicate: once you can express “is this value valid geometry” as a custom expectation, the same skeleton covers bounds, CRS-per-row and dimensionality checks.

Why a custom expectation is required

A built-in expectation operates on scalar column values — numbers, strings, dates — and has no notion of a geometry. A validity check needs to parse each cell’s WKT into a Shapely geometry and apply is_valid, which is a domain operation the framework cannot supply. Great Expectations’ extension model handles exactly this: a ColumnMapExpectation maps a boolean function over a column, and a paired metric computes the per-row result. Writing both is what lets a spatial predicate live inside a declarative suite alongside the built-in attribute checks.

Component reference

Step-by-step implementation

The example targets Great Expectations 0.18+ and Shapely 2.x, and produces expect_column_values_to_be_valid_geometry.

Step 1 — Define the Shapely-backed metric

The metric maps a parsing-and-validity function over the column, returning a boolean Series. It must never raise on bad input — an unparseable cell resolves to False.

from shapely import from_wkt, is_valid
from great_expectations.expectations.metrics import (
    ColumnMapMetricProvider, column_condition_partial,
)
from great_expectations.execution_engine import PandasExecutionEngine


def _valid_wkt(value: str) -> bool:
    try:
        return bool(is_valid(from_wkt(value)))
    except Exception:
        return False


class ColumnValuesAreValidGeometry(ColumnMapMetricProvider):
    condition_metric_name = "column_values.valid_geometry"

    @column_condition_partial(engine=PandasExecutionEngine)
    def _pandas(cls, column, **kwargs):
        return column.map(_valid_wkt)

Step 2 — Define the expectation and link the metric

from great_expectations.expectations.expectation import ColumnMapExpectation


class ExpectColumnValuesToBeValidGeometry(ColumnMapExpectation):
    """Expect each WKT value in the column to parse to a valid geometry."""

    map_metric = "column_values.valid_geometry"
    success_keys = ("mostly",)          # allow a tolerance, e.g. mostly=1.0

Step 3 — Provide examples that double as tests and docs

    examples = [{
        "data": {"wkt": [
            "POLYGON((0 0,1 0,1 1,0 1,0 0))",     # valid
            "POLYGON((0 0,1 1,1 0,0 1,0 0))",     # self-intersecting -> invalid
        ]},
        "tests": [{
            "title": "basic_validity",
            "include_in_gallery": True,
            "in": {"column": "wkt", "mostly": 1.0},
            "out": {"success": False, "unexpected_index_list": [1]},
        }],
    }]

Step 4 — Register and use it in a suite

Importing the module registers the expectation; then it is available on a batch like any built-in.

import great_expectations as gx
import expect_valid_geometry            # noqa: F401 — import registers it

ctx = gx.get_context()
batch = ctx.data_sources.pandas_default.read_dataframe(df)
batch.expect_column_values_to_be_valid_geometry("wkt", mostly=1.0)

Verification pattern

Great Expectations runs an expectation’s own examples as its test suite, so the fastest verification is the framework’s diagnostic runner, which exercises every declared example and reports coverage.

python -c "from expect_valid_geometry import ExpectColumnValuesToBeValidGeometry as E; \
print(E().run_diagnostics().to_dict()['tests'])"
# Each declared test should report a passing result

Failure modes and edge cases

1. Metric that raises. If _valid_wkt lets an exception escape, one malformed cell aborts the batch; catch and return False so the row is simply unexpected.
2. mostly misused. Setting mostly=0.99 silently tolerates 1% invalid geometry; use 1.0 for a hard gate unless partial validity is genuinely acceptable.
3. Empty geometry. from_wkt("POLYGON EMPTY") parses and is technically valid, so an emptiness rule needs its own predicate, not the validity one.
4. WKB vs WKT columns. A column of WKB bytes needs from_wkb; a validity expectation written for WKT silently fails every row on a WKB column.
5. CRS ignored. Validity says nothing about coordinate system; pair this expectation with a CRS check, as the attribute schema validation pattern does.

Conclusion

A custom geometry expectation is a metric plus an expectation class: a Shapely-backed per-row function that never raises, wrapped as a ColumnMapExpectation with examples that serve as both tests and documentation. With that skeleton, any spatial predicate becomes a first-class rule in a declarative suite. For the broader suite design, return to Great Expectations spatial expectations.

Related

• Great Expectations Spatial Expectations
• Validating Attribute Schemas with Great Expectations
• Parent: Great Expectations Spatial Expectations