Testing Nodata and Mask Handling in rasterio

8 min read

A raster mask decides which cells count. Everything downstream of it — statistics, overlays, exports, reprojections — is computed over whichever population the mask defines, so a mask that is wrong produces answers that are internally consistent and externally false. This guide sits beneath raster and grid assertions and shows how to test masking with rasterio 1.3+ directly: asserting that the declared nodata is actually present, that the mask’s polarity is the one the code assumes, that float sentinels compare exactly, and that an alpha band and a nodata value do not disagree.

The defect worth naming up front is polarity. NumPy’s masked arrays use True to mean masked out; rasterio’s dataset_mask uses 255 to mean valid. Code that mixes the two conventions produces a mask that is exactly inverted, and an inverted mask does not error — it silently computes every statistic over the fill instead of over the data.

Root cause: four ways a raster says “no value here”

Rasterio exposes several mechanisms for the same idea, and they are not equivalent. Knowing which one a dataset uses is a prerequisite for testing it, because an assertion written against the wrong mechanism passes vacuously.

A declared nodata value is a single number in the metadata. Cells equal to it are fill. It is the most common mechanism, the cheapest to check, and the one that fails when the value is declared but never written.

An alpha band is an extra band whose values encode per-cell validity or transparency. It carries more information than a sentinel — partial validity is expressible — and it is independent of the data bands’ values, so no legitimate measurement can be mistaken for fill.

An internal mask band is a separate per-band or per-dataset mask stored inside the file. It is the most explicit mechanism and the least commonly present.

No mechanism at all is a real state and a dangerous one. The raster has fill cells with some conventional value that nothing declares, so every consumer must know the convention out of band, and one that does not will treat the fill as data.

Four mechanisms, four failure modes Four rows describing the mechanisms a raster can use to mark cells as having no value. A declared nodata value marks any cell equal to a sentinel number; it expresses only a binary valid or invalid state, and its characteristic failure is being declared in the metadata while never actually appearing in the array, so masking is never exercised. An alpha band encodes validity per cell independently of the data values, can express partial validity, and characteristically fails by disagreeing with a nodata value declared alongside it. An internal mask band is the most explicit mechanism and characteristically fails by being absent when code assumes its presence. Having no mechanism at all places the convention outside the file entirely, so any consumer unaware of it treats the fill as measurement. MECHANISM EXPRESSES HOW IT FAILS Declared nodata value a single sentinel number valid or invalid, nothing between declared but never written — masking is never exercised Alpha band a separate validity band partial validity, per cell disagrees with a nodata value declared alongside it Internal mask band stored in the file explicit per-band validity absent when code assumes it exists Nothing declared a convention, held elsewhere nothing the file can state a consumer unaware of the convention treats fill as data

Mask reference

Call Returns Convention Use for
src.read(1, masked=True) Masked array True = masked out Statistics over valid cells
src.dataset_mask() uint8 array 255 = valid Whole-dataset validity
src.read_masks(1) uint8 array 255 = valid Per-band validity
src.nodata Scalar or None — Asserting the declaration
arr.mask Boolean array True = masked out Comparing mask shape
arr.count() Integer — The contributing population

The polarity column is the whole reason for the table. Two of these use True-means-invalid and two use 255-means-valid, and code that converts between them without thinking produces an inverted mask that computes over exactly the wrong cells.

Step-by-step implementation

The assertions target rasterio 1.3+ and NumPy.

Step 1 — Assert the declaration and the presence

Both halves are required. A declared value that never appears means masking is untested; a value that appears without being declared means it is being counted as data.

import rasterio
import numpy as np

def test_nodata_is_declared_and_present(raster_path):
    with rasterio.open(raster_path) as src:
        assert src.nodata is not None, "no nodata declared — fill would be read as data"
        raw = src.read(1)
        present = int(np.count_nonzero(raw == src.nodata))
    assert present > 0, (
        f"nodata {src.nodata} is declared but appears in 0 cells — "
        "masking is never exercised by this fixture"
    )

Step 2 — Pin the polarity explicitly

Never convert between conventions implicitly. Convert once, in a named helper, and assert the conversion in its own test.

def valid_mask(src, band: int = 1) -> np.ndarray:
    """Boolean array where True means the cell carries a real measurement."""
    return src.read_masks(band) == 255        # rasterio: 255 = valid

def test_mask_polarity(raster_path):
    with rasterio.open(raster_path) as src:
        valid = valid_mask(src)
        arr = src.read(1, masked=True)
    # numpy masked arrays use True = masked OUT, so the two must be inverses.
    assert np.array_equal(valid, ~arr.mask), "mask polarity conventions disagree"

That equality is the assertion that catches an inverted mask, and it is worth writing even when nothing currently converts between the two — because the day something does, this is the test that fails instead of a statistic quietly halving.

Step 3 — Compare float sentinels with a tolerance

A nodata of -9999.0 declared as float64 and stored as float32 may not compare exactly equal, leaving a handful of fill cells unmasked. The masking rasterio performs handles this correctly; a hand-rolled comparison frequently does not.

def test_float_sentinel_masks_every_fill_cell(dem_path):
    with rasterio.open(dem_path) as src:
        arr = src.read(1, masked=True)
        raw = src.read(1)
        near_fill = np.isclose(raw, src.nodata, rtol=0, atol=1e-3)
    unmasked_fill = int(np.count_nonzero(near_fill & ~arr.mask))
    assert unmasked_fill == 0, (
        f"{unmasked_fill} cells are within 1e-3 of the nodata value but were not masked — "
        "a float precision mismatch between the declared and stored sentinel"
    )

Step 4 — Assert alpha and nodata agree

When both mechanisms are present they are two claims about the same thing, and nothing forces them to match.

def test_alpha_and_nodata_agree(rgba_path):
    with rasterio.open(rgba_path) as src:
        if src.count < 4 or src.nodata is None:
            import pytest; pytest.skip("no alpha band or no declared nodata")
        alpha_valid = src.read(4) > 0
        sentinel_valid = src.read(1) != src.nodata
    disagreements = int(np.count_nonzero(alpha_valid != sentinel_valid))
    assert disagreements == 0, f"{disagreements} cells where alpha and nodata disagree"
An inverted mask computes over the fill Two computations of the same statistic differing only in mask polarity. With the correct polarity the mean is taken over the eight thousand four hundred cells carrying measurements and reports a plausible value with a normal spread. With the polarity inverted, the same call computes over the sixteen hundred fill cells instead; because the fill is one repeated sentinel the mean equals that sentinel exactly and the standard deviation is exactly zero. A closing note records that a standard deviation of precisely zero on a continuous band is the clearest available symptom of an inverted mask, and that neither computation raises an error. correct polarity computed over 8 400 measurement cells mean = 14.2 std = 3.8 a plausible measurement with a normal spread polarity inverted computed over 1 600 fill cells mean = −9999.0 std = 0.0 one repeated value — and no error raised A standard deviation of exactly zero on a continuous band is the clearest available symptom of an inverted mask. Asserting a non-zero spread costs one line and catches the whole class.

Verify the fix

Invert the polarity deliberately once and confirm the suite fails:

pytest -q tests/test_masks.py -v

The polarity test should fail with its own message, and the statistics tests should fail with implausible values. If the statistics pass while the polarity test fails, the statistics are not using the mask at all — which is a third defect worth knowing about.

Choosing a mechanism for new outputs

When a pipeline writes a raster, the mechanism it uses to mark fill is a decision worth making rather than inheriting from a default. Three considerations settle it.

Can a legitimate measurement equal the sentinel? If yes, a nodata value is unusable — the fill and the data are indistinguishable — and an alpha or mask band is required. This is why a sentinel of 0 is dangerous for any quantity that can legitimately be zero, and why -9999 is conventional for elevation and useless for a signed difference raster.

Does partial validity need to be expressed? A cell that is half-covered by a resampled input has a meaningful intermediate alpha value and no meaningful sentinel. Only the alpha mechanism expresses it.

What will consumers actually read? A mask mechanism nobody downstream honours is worse than none, because it creates a false sense of coverage. Choosing the mechanism the consuming tools support is more important than choosing the most expressive one.

Choosing the fill mechanism for a new output Three sequential questions. The first asks whether a legitimate measurement could equal the sentinel value; if it could, a declared nodata value is unusable because fill and data become indistinguishable, so an alpha or mask band is required. The second asks whether partial validity must be expressed, such as a cell only half covered by a resampled input; if so, only an alpha band can carry that information. The third asks whether the downstream consumers actually honour the chosen mechanism, since a mask nothing reads creates false confidence about coverage. If none of the three forces a change, a declared nodata value is the simplest sufficient choice. 1 · Can a real measurement equal the sentinel? zero for a difference raster; −9999 for a depth yes → a nodata value is unusable fill and data are indistinguishable; use alpha or a mask band 2 · Must partial validity be expressed? a cell half covered by a resampled input yes → only an alpha band carries it a sentinel is binary by construction 3 · Will consumers honour it? check the tools, not the specification no → use what they do read an unread mask creates false confidence about coverage If none of the three forces a change, a declared nodata value is the simplest mechanism that is sufficient — and the easiest to assert.

Failure modes and edge cases

  1. A sentinel that is a legitimate value. Zero as nodata for a difference raster masks every cell where the two inputs agreed, which is usually most of them. Choose a sentinel outside the physically possible range.
  2. Mask polarity flipped in a helper. The most common instance is negating a rasterio mask to feed NumPy without checking, which produces statistics over the fill. The polarity test above is the only thing that catches it before a number is questioned.
  3. A fully-masked band. Statistics over an entirely masked array return masked values rather than numbers, and comparisons against them are neither true nor false. Assert a non-zero valid count first.
  4. NaN treated as fill. NaN is not the declared nodata and is not masked by it, but it propagates through every statistic. Check finiteness separately, and decide deliberately whether NaN should be masked.
  5. An alpha band on a dataset that also declares nodata. Two mechanisms means two claims and no arbiter. Assert they agree, or remove one.
  6. Masks lost on write. A driver that does not support an internal mask silently drops it, so a round trip loses coverage information that every previous test confirmed was correct. Read the artefact back and assert the mask survived.

Conclusion

A mask is a claim about which cells count, and it is testable independently of the values it governs. Asserting that the declared fill is present, pinning the polarity in one place with a test that proves the conversion, comparing float sentinels with a tolerance, and reconciling alpha against nodata turns the mask from an assumption into a contract — the coverage half of raster and grid assertions.

Record the mask’s valid-cell count in the run summary alongside the statistics it governs, so a coverage change is visible as a number rather than inferred from a value that moved.