Fix pseudo-irregular time series direct reads

[krowvin]

Summary

Fixes the time-series direct-read path for pseudo-irregular old-style local-regular IDs such as *.~15Minutes.* when X-CWMS-LRTS-Formatting: true is present.

Root cause

The direct-read path could resolve pseudo-irregular old-style IDs through new LRTS formatting, producing local-regular metadata and returning no values. It also treated ~ intervals as regular without first honoring interval_utc_offset = UTC_OFFSET_IRREGULAR, which could incorrectly invoke gap-fill behavior for pseudo-irregular data.

Changes

• Detect old-style ~ IDs whose old-format metadata has irregular interval offset.
• Resolve/read those IDs under old LRTS formatting for direct reads.
• Suppress regular gap-fill and report zero interval duration for truly irregular offset metadata.
• Add an integration regression test that seeds a pseudo-irregular ~15Minutes series and verifies the CDA response matches retrieve_ts under the LRTS formatting header.

Validation

• ./gradlew.bat :cwms-data-api:compileJava :cwms-data-api:compileTestJava --no-daemon
• ./gradlew.bat :cwms-data-api:integrationTests --tests "*TimeSeriesDirectReadParityIT.pseudoIrregularReadWithLrtsHeaderMatchesRetrieveTs" "-PCDA_POOL_INIT_SIZE=5" "-PCDA_POOL_MAX_ACTIVE=10" "-PCDA_POOL_MAX_IDLE=5" "-PCDA_POOL_MIN_IDLE=2" --no-daemon
• ./gradlew.bat :cwms-data-api:checkstyleMain :cwms-data-api:checkstyleTest --no-daemon

Note: broader TimeSeriesDirectReadParityIT runs were not clean in this local container due fixture/setup issues before the relevant endpoint path was exercised, including Oracle seed partition check failures and a default-data county error.

Closes #1745

[MikeNeilson]

I'm not sure why this cares about the LRTS formatting. There is a function that just tells you:

https://github.com/HydrologicEngineeringCenter/cwms-database/blob/648b849254db226bedd9d9a38e94628df28ec9d5/schema/src/cwms/cwms_ts_pkg.sql#L1064

use JOOQ to call (may already be a helper somewhere in CDA that caches the response.

Basic logic:

1. Is LRTS -> build expected times and map it out
2. Is not LRTS -> just return the data.

[MikeNeilson]

you can build and return a JOOQ query. keeps the type safety.

NOTE: some of those elements will still need to be text.

NOTE2: since we have to do things in a loop anyways to build the output, it will likely be faster to do the normalization of the quality_code in Java. Moving between plsql and sql does cause a context switch which has some performance concerns.

NOTE3: av_tsv_dqu already handles the unit conversions (that's what the U stands for in dqu), you just put units_id = ... in the where clause. Additional conversions are no-ops, but force the context switch.

[krowvin]

pseudo-irregular ~15Minutes series were being handled like true local-regular series in the direct-read path, which caused incorrect LRTS/session behavior and large window reads

changes

• This fix uses database authoritative LRTS detection via CWMS_TS_PACKAGE.call_IS_LRTS__2(...).
• Pseudo-irregular series with interval_utc_offset == UTC_OFFSET_IRREGULAR are treated as irregular and return observed rows only.
• True local-regular/LRTS series still use expected-time generation and gap-fill behavior.
• Removed the old workaround that forced old LRTS formatting through recursive DAO construction and raw JDBC row reads.
• Kept the direct row read in jOOQ against AV_TSV_DQU.
• Row reads now filter by requested UNIT_ID and normalize quality in Java.
• Added/adapted regression coverage for ITPARPIRR.Flow.Inst.~15Minutes.0.BENCH with X-CWMS-LRTS-Formatting: true.
• Verified the pseudo-irregular regression test passes.
• Verified full TimeSeriesDirectReadParityIT passes locally.
• Verified compile and checkstyle tasks pass locally with JDK 21.
• Tested MVP timeseries Baldhill_Dam.Flow-Out.Inst.~15Minutes.0.best.

tests

• For a 70,177-row production sized window (2024-01-01 to 2026-01-01), local CDA returned the same row count, timestamps, qualities, and pseudo-irregular metadata.
• Local timing for that 70k-row window was about 2 seconds, compared with production runs around 29-38 seconds.
• pulled data from each instance, local and prod, to confirm no drift in datasets.

timings

For this command

localhost

curl -sS -w '\nstatus=%{http_code} time=%{time_total}s bytes=%{size_download}\n' \
  -H 'Accept: application/json;version=2' \
  -H 'X-CWMS-LRTS-Formatting: true' \
  'http://localhost:8081/cwms-data/timeseries?name=Baldhill_Dam.Flow-Out.Inst.~15Minutes.0.best&office=MVP&unit=cfs&begin=2025-05-01T00:00:00Z&end=2025-06-01T00:00:00Z&page-size=100000'

national

curl -sS -w '\nstatus=%{http_code} time=%{time_total}s bytes=%{size_download}\n' \
  -H 'Accept: application/json;version=2' \
  'https://cwms-data.usace.army.mil/cwms-data/timeseries?name=Baldhill_Dam.Flow-Out.Inst.~15Minutes.0.best&office=MVP&unit=cfs&begin=2025-05-01T00:00:00Z&end=2025-06-01T00:00:00Z&page-size=100000'

(LRTS header not in Prod yet)

Target	Window	Rows	Runs	Min	Median	Mean	Max	Next Page
Local CDA	2024-01-01 to 2026-01-01	70,177	5	1.815s	1.903s	2.075s	2.668s	No
Production CDA	2024-01-01 to 2026-01-01	70,177	3	29.088s	37.805s

[MikeNeilson]

Note possible conflicts with #1780 depending on which gets merged in first.

[MikeNeilson]

I'm still not sure why the "Old formatting/New Formatting" methods need to exist.

The method of formatting on output and input expectation is set at the session level and is purely visual. Regardless of visual style a time series is either: regular, irregular, psuedo regular (effective irregular for this purpose), or local regular.

That status of the time series determine the 2 modes of data retrieval, none of which has anything to do with with an LRTS timeseries is rendered as "1DayLocal" or "~1Day" for the interval. The database output of the TS time will render that correctly.

It reads like a giant red-herring that detracts from the actual requirement of the read which boils down to the following:

`Do I need to provided predictable time placeholders (regular, local regular) or just render the rows as stored in the database (irregular, psuedoregular).

[MikeNeilson]

TSV PR was merged in, conflicts need to be fixed. Also see my comments about LRTS methods and usage, feels like something is stuck down a rabbit hole of over thinking and needs to get dug out.

Ask database if this TSID is LRTS or REGULAR interval
yes -> have expected interval set in some way shape or form
no -> just return the data from the tsv query as it, the timestamps provided are by definition correct.

[MikeNeilson]

Still some concerns with the overall logic, but some of that wasn't touched in this PR.

Actual concerns address in specific comments.

Otherwise, looks reasonable, and the tests show expected behavior correctly.

[MikeNeilson]

why are we spending time formatting a timestamp that can be used as-is in the query?

[MikeNeilson]

These (START_DATE and END_DATE) likely can be ignored, the query itself handles the situation better.

However, if they are to be used, their purpose is to bracket which physical at_tsv table is used.

it should be start >= "year of data-01-01" end <= "year+1-01-01", it doesn't need the hours, minutes, or seconds.

[MikeNeilson]

(isLrts && isLocalRegularInterval(intervalPart)

By definition these are either both true, or both false. You've already asked the database if the time series is local regular. We should not be trying to duplicate the logic in CDA.

[krowvin]

Spoke in CDA biweekly if RMA would also be able to review and ensure nothing was missed.