Fix pseudo-irregular time series direct reads#1781
Conversation
| String cursor = null; | ||
| Timestamp tsCursor = null; | ||
|
|
||
| if (forceOldLrtsFormatting) { |
There was a problem hiding this comment.
I'm not sure why this cares about the LRTS formatting. There is a function that just tells you:
use JOOQ to call (may already be a helper somewhere in CDA that caches the response.
Basic logic:
- Is LRTS -> build expected times and map it out
- Is not LRTS -> just return the data.
| } | ||
|
|
||
| private static String buildVersionedRowsSql(boolean includeEntryDate) { | ||
| return "select date_time," |
There was a problem hiding this comment.
you can build and return a JOOQ query. keeps the type safety.
NOTE: some of those elements will still need to be text.
NOTE2: since we have to do things in a loop anyways to build the output, it will likely be faster to do the normalization of the quality_code in Java. Moving between plsql and sql does cause a context switch which has some performance concerns.
NOTE3: av_tsv_dqu already handles the unit conversions (that's what the U stands for in dqu), you just put units_id = ... in the where clause. Additional conversions are no-ops, but force the context switch.
Summary
Fixes the time-series direct-read path for pseudo-irregular old-style local-regular IDs such as
*.~15Minutes.*whenX-CWMS-LRTS-Formatting: trueis present.Root cause
The direct-read path could resolve pseudo-irregular old-style IDs through new LRTS formatting, producing local-regular metadata and returning no values. It also treated
~intervals as regular without first honoringinterval_utc_offset = UTC_OFFSET_IRREGULAR, which could incorrectly invoke gap-fill behavior for pseudo-irregular data.Changes
~IDs whose old-format metadata has irregular interval offset.~15Minutesseries and verifies the CDA response matchesretrieve_tsunder the LRTS formatting header.Validation
./gradlew.bat :cwms-data-api:compileJava :cwms-data-api:compileTestJava --no-daemon./gradlew.bat :cwms-data-api:integrationTests --tests "*TimeSeriesDirectReadParityIT.pseudoIrregularReadWithLrtsHeaderMatchesRetrieveTs" "-PCDA_POOL_INIT_SIZE=5" "-PCDA_POOL_MAX_ACTIVE=10" "-PCDA_POOL_MAX_IDLE=5" "-PCDA_POOL_MIN_IDLE=2" --no-daemon./gradlew.bat :cwms-data-api:checkstyleMain :cwms-data-api:checkstyleTest --no-daemonNote: broader
TimeSeriesDirectReadParityITruns were not clean in this local container due fixture/setup issues before the relevant endpoint path was exercised, including Oracle seed partition check failures and a default-data county error.Closes #1745