Skip to content

Efficient Multiple Site Requests #6

Description

@ktarbet

Context

In applications like HEC-DSSVue, users may need to perform actions for all sites within a given state.

Current Implementation

Action Current Method Efficiency
Metadata Lookups (e.g., period of record) Batched across multiple sites in large chunks Efficient
Data Downloads Requires separate, individual queries for each time-series Inefficient

Describe the solution you'd like

Implement batched data downloads across multiple sites, using a large-chunk method similar to how we currently handle metadata lookups.

Data Suitability for Cross-Site Batching

Data Type Example Suitability Rationale
Sparse Data Annual Max Discharge High The limited number of values (one per year) allows multiple sites to fit within request limits.
Continuous Data 5-Minute Intervals Low A single site can easily exceed the USGS 50,000 record limit on its own, making cross-site batching less practical
Daily Data 1-Day Intervals Medium A single time-series 100 years long approaches the limit without chunking across time; multiple time-series queries could be investigated.

Note: We have already implemented pagination/chunking to handle continuous data limits within a single time-series (Site/Parameter/Statistic).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions