rzmk/ckan_geoconnex_bulk_runner
1
0
Fork 0
mirror of https://github.com/dathere/ckan_geoconnex_bulk_runner.git synced 2026-06-20 19:32:56 +00:00
Code Issues Projects Releases Packages Wiki Activity
📦 (WIP) Program that sets up a bulk integration between a CKAN water data hub's datasets and vector geospatial features with Geoconnex. Ran as a Docker container and built with Rust.
23 commits 1 branch 1 tag 157 KiB
  • Rust 98%
  • Dockerfile 2%
Find a file
Exact
rzmk 59564c9de3 build: include ca-certificates in container
2026-06-15 13:02:51 -04:00
.github/workflows ci: use default Git context 2026-06-15 11:48:13 -04:00
bulk_loader build: include ca-certificates in container 2026-06-15 13:02:51 -04:00
ckan_geoconnex_bulk_runner_py feat: enhanced cargo workspace, NM usage, Dockerfile 2026-06-15 11:20:51 -04:00
generate_release feat: enhanced cargo workspace, NM usage, Dockerfile 2026-06-15 11:20:51 -04:00
geoconnex_utils fix: use nmwdh for JSON-LD construction 2026-06-15 11:31:15 -04:00
.gitignore refactor: convert to cargo workspace setup 2026-05-20 11:34:25 -04:00
Cargo.lock feat: enhanced cargo workspace, NM usage, Dockerfile 2026-06-15 11:20:51 -04:00
Cargo.toml feat: enhanced cargo workspace, NM usage, Dockerfile 2026-06-15 11:20:51 -04:00
README.md docs: fix bold markup 2026-06-15 11:21:19 -04:00

README.md

ckan_geoconnex_bulk_runner

https://github.com/user-attachments/assets/779fe866-d511-44f3-91a9-a1c2e1cfa189

Status: This codebase is currently a work in progress and more documentation is planned.

The ckan_geoconnex_bulk_runner codebase is part of a multi-service infrastructure to sync water data hubs using CKAN to the Geoconnex knowledge graph.

  • geoconnex_utils: Helper functions used throughout the ckan_geoconnex_bulk_runner project including JSON-LD construction and JSON schema validation.
  • geoconnex_release: Compatible CKAN datasets and vector geospatial features for all connected water data hubs are uploaded to a ckan-geoconnex-web-resources.jsonl file in the latest GitHub release.
  • bulk_loader: Requests and outputs the latest JSONL file from the latest GitHub release. This is ran as a Docker container by Geoconnex on a periodic frequency to upload all water data hub web resources to the Geoconnex knowledge graph following the Geoconnex bulk contribution specification.
  • ckan_geoconnex_bulk_runner_py: Python library intended for usage by the ckanext-gztr and DataPusher+ CKAN extensions.

This runner is expected to be implemented for a water data hub with the relevant fields and/or ckanext-gztr (not open-source yet) and/or DataPusher+ enabled. For questions reach out to datHere, Center for Geospatial Solutions, or add an issue/discussion.