Shapely vs PostGIS for In-Pipeline Topology Checks

3 min read

A topology rule — no self-intersections, no gaps, no overlaps — can be enforced in-process with Shapely or pushed to PostGIS, and the right choice depends on where the data lives, how large it is, and whether the same rule must also gate ingestion. This comparison sits beneath choosing spatial testing tools and works through the decision on the axes that decide it in practice: memory footprint, whether the check must also run as a database constraint, cross-engine agreement between GEOS builds, and raw throughput. Both call into GEOS for the actual predicates, so the question is rarely correctness of a single check — it is where the check belongs in the pipeline.

The root difference: where the data already is

Shapely operates on geometries already in Python memory, so an in-process check is free of a database round-trip but bounded by RAM — loading a million-row layer to validate it can exhaust the runner. PostGIS operates where the data is stored, so a server-side check scales past memory and can double as an ingestion constraint that stops bad geometry from ever landing, at the cost of needing a database in the test environment. Because both delegate to GEOS, a rule expressed in each should agree — but only if the two GEOS builds match, which is exactly the pinning problem the containerized runtimes work addresses.

Comparison reference

Where Shapely wins

When the test already holds the data as a GeoDataFrame and the fixture fits in memory, in-process Shapely is simplest and fastest — no database to stand up, no round-trip, and the failing geometry is right there to serialize into an artifact. It is the natural fit for the fast pre-merge tier.

# Shapely / GeoPandas: in-memory topology validation
import geopandas as gpd
from shapely import is_valid

gdf = gpd.read_file("tests/fixtures/parcels.gpkg")
invalid = gdf[~gdf.geometry.map(is_valid)]
assert invalid.empty, f"{len(invalid)} invalid geometries"

# no-overlap check via a self-spatial-join on the in-memory frame
pairs = gpd.sjoin(gdf, gdf, predicate="overlaps")
assert pairs.empty, "overlapping parcels detected"

Where PostGIS wins

When the data lives in PostGIS, is too large to load, or the identical rule must gate ingestion so invalid geometry cannot persist, the server-side check wins. Pushing the predicate to the database avoids moving millions of rows into Python and lets the same SQL run both as a test and as a production constraint — the enforcing no-gaps/no-overlaps with PostGIS pattern shows the ingestion-gate form.

-- PostGIS: validity + overlap gate that also scales past memory
SELECT id FROM parcels WHERE NOT ST_IsValid(geom);

SELECT a.id, b.id
FROM parcels a
JOIN parcels b ON a.id < b.id AND ST_Overlaps(a.geom, b.geom);

Drive it from the same suite with psycopg2 so database and in-process checks share one test run:

import psycopg2
with psycopg2.connect(dsn) as conn, conn.cursor() as cur:
    cur.execute("SELECT count(*) FROM parcels WHERE NOT ST_IsValid(geom);")
    (bad,) = cur.fetchone()
    assert bad == 0, f"{bad} invalid geometries in DB"

Keeping both consistent

The one real hazard in mixing the two is cross-engine disagreement: PostGIS and your local Shapely wheel can link different GEOS builds and return different results at the boundary of a predicate. Pin both to the same GEOS version and add a cross-engine test that runs a known edge case through each and asserts they agree, so a divergence surfaces in CI rather than as a “passes locally, fails in the database” report.

Failure modes and edge cases

1. Loading a huge layer into Shapely. Reading a production-scale table into a GeoDataFrame to check validity OOMs the runner; push the check to PostGIS.
2. GEOS build mismatch. Different GEOS behind PostGIS and Shapely diverge on boundary-touching predicates; pin both and cross-check.
3. In-process check that should gate ingestion. Validating only in Python leaves the database ingestion path unguarded, so bad geometry still lands from other writers.
4. sjoin self-overlap O(n²) blowup. A self-spatial-join without an index degrades badly on large in-memory frames; ensure the frame is spatially indexed or move the join to a GiST-backed query — see R-tree vs GiST.
5. Snap-to-grid slivers. Both engines can emit sub-centimetre overlap fragments after union/difference; guard overlap assertions with an area floor so artifacts do not read as defects.

Conclusion

Shapely and PostGIS both delegate topology to GEOS, so the decision is about placement, not correctness: in-process Shapely when the data is in memory and the fixture is small, PostGIS when the data lives in the database, is too large to load, or the rule must also gate ingestion. Pin GEOS on both sides and cross-check the boundary cases, and the two stay consistent. For the wider tool-selection framework, return to choosing spatial testing tools.

Related

• pytest-geo vs Great Expectations for Spatial Validation
• R-tree vs GiST Index Performance in Test Environments
• Parent: Choosing Spatial Testing Tools