Skip to content

DIE-306: Richer pygeoapi layer metadata (discovery pilot) - #144

Open
chasetmartin wants to merge 2 commits into
mainfrom
DIE-306-adr-investigate-populating-layer-metadata-and-field-descriptions
Open

chasetmartin wants to merge 2 commits into
mainfrom
DIE-306-adr-investigate-populating-layer-metadata-and-field-descriptions

Conversation

@chasetmartin

@chasetmartin chasetmartin commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Discovery task for DIE-306: how much richer metadata can we get in front of users in GIS clients? Piloted on nm_waterlevel_trends, then checked in QGIS against the sample layers.

Improves layer metadata overall

  • Abstract: a 6-sentence abstract (feature, measurement and units, sources, method, update cadence, main caveat) replaces the one-line description.
  • Keywords: product-specific keywords on top of the shared three.
  • Links: a data dictionary link (rel: describedby → new docs/products/README.md#<output_type>) and a per-collection CC-BY 4.0 license link.
  • Everything else unchanged: the other 30 products look exactly as before until they get an abstract (Phase 2). description itself is unchanged, so Dagster and the GeoJSON files aren't affected.

What QGIS shows: title, full abstract, keywords, license (Access → Licenses) and both links. Field descriptions don't reach QGIS: pygeoapi can't set them from config, and QGIS doesn't read /schema. The README link covers that for now.

Found along the way (out of scope)

  • Point layers mix 2D and 3D geometries (elevation stored as Z), which triggers a QGIS warning.
  • QGIS types columns from the first features it downloads, so a sparse numeric column like well_depth shows as String.

Testing

  • New checks in tests/test_pygeoapi_config.py and a new tests/test_product_docs.py: README sections exist, method text matches the code, sample columns are documented.
  • Note: no CI workflow runs pytest on this branch or on PRs to main (this predates the PR), so run these locally: uv run --extra parquet --with jinja2 --with geopandas pytest tests/test_pygeoapi_config.py tests/test_product_docs.py.
  • Ran locally against ~/pygeoapi-test with no layer generation (render command in SPEC §6.4).

The data dictionary link points at main, so it resolves after merge.

…trends

Products with an abstract get it as their pygeoapi description, plus
product keywords, a data dictionary link (docs/products/README.md) and a
per-collection license link. The Docker build now validates the config.
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Your pull request is automatically being deployed to Dagster Cloud.

Location Status Link Updated
die-orchestration View in Cloud Oct 15, 2026 at 08:14 PM (UTC)

Keep the Dockerfile unchanged; the config tests cover the new links.
@chasetmartin

Copy link
Copy Markdown
Contributor Author

Nice hover effect on layer:
Screenshot 2026-10-15 at 1 15 53 PM
Metadata tab, with link to the data product license (creative commons - we can talk about that?) and a data dictionary housed as a readme in the repo.
Screenshot 2026-10-15 at 1 16 25 PM
Richer description on pygeoapi itself:
Screenshot 2026-10-15 at 1 17 41 PM

@chasetmartin

Copy link
Copy Markdown
Contributor Author

This does not account for ArcGIS Pro, which I will have to test with Cris or another user.

Comment thread docs/products/README.md

**Products:** `nm_waterlevel_trends`

One point feature per well that has depth-to-water measurements, labeled with

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it depth to water or depth to water below ground surface?

Comment thread docs/products/README.md
Comment on lines +65 to +66
| `first_observation_datetime` | Date of the first daily value (UTC, ISO 8601) | | |
| `last_observation_datetime` | Date of the last daily value (UTC, ISO 8601) | | |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a fan of using earliest... and latest.... I think first... works pretty well, but last... indicates to me that there will be no more observations.

@chasetmartin chasetmartin Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree latest or most_recent is probably better. This is a code change for more than one layer so I created a ticket so we can take care of it before the API launch.

description: "Per-well depth-to-water trend (slope and category), all NM sources"
abstract: >-
One feature per well with depth-to-water measurements, labeled with the direction
and rate of change of its water level. Each well's record is reduced to one value

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and rate of change of its water levels between the first and last known observations.

(if a different nomenclature like latest is adopted then change "lat" to "latest")

Comment on lines +76 to +78
labeled "not enough data". Sources: NMBGMR, USGS NWIS, the Water Quality Portal,
NM OSE, the Interstate Stream Commission, Bernalillo County, the City of
Albuquerque, EBID and PVACD. Updated monthly. The trend is in depth to water, so

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the Water Quality Portal and Interstate Stream Commission are called WQP and ISC elsewhere should we abbreviate them here? Or, for the sake of consistency should we not use abbreviations?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least for NM OSE. I feel like many people know USGS, though NWIS is a bit less well known. (And we are NMBGMR so I assume people would know what it stands for, but still something to consider).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'll investigate another way to highlight sources for each layer - spelling it out in every abstract is overkill (I think). Using abbreviations in the abstract but spelling them out in another place is probably the better approach.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that spelling it out in every abstract would be overkill, but they should be enumerated and spelled out somewhere. Perhaps the landing page?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants