Skip to content

Fix pseudo-irregular time series direct reads#1781

Draft
krowvin wants to merge 1 commit into
developfrom
issue-1745-pseudo-irregular
Draft

Fix pseudo-irregular time series direct reads#1781
krowvin wants to merge 1 commit into
developfrom
issue-1745-pseudo-irregular

Conversation

@krowvin

@krowvin krowvin commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes the time-series direct-read path for pseudo-irregular old-style local-regular IDs such as *.~15Minutes.* when X-CWMS-LRTS-Formatting: true is present.

Root cause

The direct-read path could resolve pseudo-irregular old-style IDs through new LRTS formatting, producing local-regular metadata and returning no values. It also treated ~ intervals as regular without first honoring interval_utc_offset = UTC_OFFSET_IRREGULAR, which could incorrectly invoke gap-fill behavior for pseudo-irregular data.

Changes

  • Detect old-style ~ IDs whose old-format metadata has irregular interval offset.
  • Resolve/read those IDs under old LRTS formatting for direct reads.
  • Suppress regular gap-fill and report zero interval duration for truly irregular offset metadata.
  • Add an integration regression test that seeds a pseudo-irregular ~15Minutes series and verifies the CDA response matches retrieve_ts under the LRTS formatting header.

Validation

  • ./gradlew.bat :cwms-data-api:compileJava :cwms-data-api:compileTestJava --no-daemon
  • ./gradlew.bat :cwms-data-api:integrationTests --tests "*TimeSeriesDirectReadParityIT.pseudoIrregularReadWithLrtsHeaderMatchesRetrieveTs" "-PCDA_POOL_INIT_SIZE=5" "-PCDA_POOL_MAX_ACTIVE=10" "-PCDA_POOL_MAX_IDLE=5" "-PCDA_POOL_MIN_IDLE=2" --no-daemon
  • ./gradlew.bat :cwms-data-api:checkstyleMain :cwms-data-api:checkstyleTest --no-daemon

Note: broader TimeSeriesDirectReadParityIT runs were not clean in this local container due fixture/setup issues before the relevant endpoint path was exercised, including Oracle seed partition check failures and a default-data county error.

Closes #1745

String cursor = null;
Timestamp tsCursor = null;

if (forceOldLrtsFormatting) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this cares about the LRTS formatting. There is a function that just tells you:

https://github.com/HydrologicEngineeringCenter/cwms-database/blob/648b849254db226bedd9d9a38e94628df28ec9d5/schema/src/cwms/cwms_ts_pkg.sql#L1064

use JOOQ to call (may already be a helper somewhere in CDA that caches the response.

Basic logic:

  1. Is LRTS -> build expected times and map it out
  2. Is not LRTS -> just return the data.

}

private static String buildVersionedRowsSql(boolean includeEntryDate) {
return "select date_time,"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can build and return a JOOQ query. keeps the type safety.

NOTE: some of those elements will still need to be text.

NOTE2: since we have to do things in a loop anyways to build the output, it will likely be faster to do the normalization of the quality_code in Java. Moving between plsql and sql does cause a context switch which has some performance concerns.

NOTE3: av_tsv_dqu already handles the unit conversions (that's what the U stands for in dqu), you just put units_id = ... in the where clause. Additional conversions are no-ops, but force the context switch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TimeSeries Endpoint - Pseudo Irregular Fixes

2 participants