Skip to content

Commit pre-built adjacency zarrs to repo for out-of-the-box routing #176

@taddyb

Description

@taddyb

Context

After #172, the adjacency zarrs contain everything needed for routing (topology + flowpath attributes). These files are small enough (<50 MB each) to live in the repository, eliminating the need for any manual data preparation before running DDR.

Current sizes (topology only):

  • hydrofabric_v2.2_conus_adjacency.zarr — 5.8 MB
  • merit_conus_adjacency.zarr — 2.3 MB

With baked-in flowpath attributes, these will grow modestly (5 float arrays × ~300-800k segments × 4 bytes ≈ 6-16 MB additional).

Proposal

  1. Rebuild both adjacency zarrs using the updated engine scripts from Bake flowpath attributes into adjacency zarr at engine build time #172
  2. Commit them to data/ (or examples/data/) in the repository
  3. Update example configs to point to the in-repo zarrs with relative paths
  4. Add to .gitattributes if needed for LFS (likely unnecessary at <50 MB)

Per-gage adjacency zarrs

The *_gages_conus_adjacency.zarr files are ~300 MB each — too large for the repo. These are only needed for gage-based training/testing, not for routing. They should remain a separate download (future ddr fetch or manual).

Files to modify

  • data/ or examples/data/ — add zarr stores
  • examples/merit/example_config.yaml — update paths
  • examples/lynker_hydrofabric/example_config.yaml — update paths
  • config/templates/*.yaml — update default paths

Acceptance criteria

  • Both adjacency zarrs committed and under 50 MB each
  • ddr route --config-name example resolves the adjacency zarr without any downloads
  • .gitignore updated if needed to not exclude these specific zarrs
  • Repo clone size remains reasonable

Depends on

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions