Add tests.analytics.parquet_indices toggle for analytics-engine IT coverage

[ahkcs]

Summary

Adds a small, opt-in test infrastructure slice so the PPL integration test suite can run end-to-end against the analytics-engine backend without per-test rewiring. Default behavior is unchanged: with the flag unset, every IT runs through the existing V2 / Calcite path exactly as before.

The single knob is -Dtests.analytics.parquet_indices=true. When set, TestUtils.createIndexByRestClient backs every test-created index with single-shard composite/parquet storage. RestUnifiedQueryAction.isAnalyticsIndex (post-#5432) reads those settings and routes the query to the analytics-engine planner (DataFusion). No additional cluster setting, REST short-circuit, or routing override — production routing is the single source of truth.

Commits

1. Add tests.analytics.parquet_indices test toggle — the infra slice (3 files, +68 LOC):

File	Change
integ-test/.../TestUtils.java	ANALYTICS_PARQUET_INDICES_PROP constant; createIndexByRestClient injects composite/parquet settings when the flag is on; loadDataByRestClient uses refresh=true to work around LuceneCommitter.getSafeCommitInfo stub
integ-test/.../PPLIntegTestCase.java	isAnalyticsParquetIndicesEnabled() predicate for per-test branching on engine semantics
integ-test/build.gradle	Forwards -Dtests.analytics.parquet_indices to :integTestRemote

2. Branch 7 stats tests on isAnalyticsParquetIndicesEnabled — demonstrates the helper (1 file, +40/-8 LOC):

SQL-spec semantics (DataFusion follows the spec; legacy DSL returns 0)

Test	What changed
testSumWithNull	SUM of all-null returns null on analytics; legacy DSL pushdown returns 0. OR analytics into the existing isPushdownDisabled() branch
testStatsPercentileByNullValue + NonNullBucket	percentile of empty/all-null group returns null on analytics. Same OR pattern

Non-deterministic head ordering

Test	What changed
testStatsWithLimit Q1 + Q2	head 5 over a 6-bucket result picks (null,36) instead of (null,null) on analytics (DataFusion hash-bucket order differs from Calcite enumerable order). Size-only assertion via the existing pushdown branch

TDigest interpolation divergence

Test	What changed
testStatsPercentileWithNull	DataFusion's TDigest interpolates p50 to 35413 vs OpenSearch's 39225; both are valid approximations within TDigest's compression-bound error. Per-engine expected value
testStatsPercentileBySpan	Same TDigest diff on the age=30 bucket (33194 vs 39225). Coordinated with opensearch-project/OpenSearch#21584 commit 5 which fixes the upstream planner-side type mismatch so the query reaches the data assertion at all

Semantic divergence pending team decision

Test	What changed
testDisableLegacyPreferred	Under PPL_SYNTAX_LEGACY_PREFERRED=false, V2 / Calcite-DSL drops the null-age bucket (5 rows) while DataFusion keeps it (6 rows). Skipped on the analytics path via Assume.assumeFalse until the team decides which behavior is the intended new-syntax semantics.

Why no force-routing cluster setting

An earlier revision of this PR added a plugins.calcite.analytics.force_routing cluster setting plus a short-circuit in RestUnifiedQueryAction.isAnalyticsIndex. With #5432 merged (routing now keys on index.pluggable.dataformat.enabled + pluggable.dataformat=composite), that override is redundant:

1. parquet_indices=true makes every test-created index carry the right settings.
2. CalciteStatsCommandIT.init() calls enableCalcite(), so the Calcite path is active.
3. isAnalyticsIndex() returns true via the index-setting check → query routes to analytics-engine.

Verified end-to-end: removing the force-routing cluster setting / IT toggle / REST short-circuit gives the exact same pass-rate as the earlier revision.

Verification

Against an analytics-engine cluster (opensearch-project/OpenSearch#21584 with commit 5 + main rebase):

./gradlew :integ-test:integTestRemote \
  -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9300 -Dtests.clustername=runTask \
  -Dtests.analytics.parquet_indices=true \
  --tests "org.opensearch.sql.calcite.remote.CalciteStatsCommandIT"

CalciteStatsCommandIT pass-rate

Test	Before this PR	After this PR
testStatsWithLimit	FAIL	PASS
testSumWithNull	FAIL	PASS
testStatsPercentileByNullValue	FAIL	PASS
testStatsPercentileByNullValueNonNullBucket	FAIL	PASS
testStatsPercentileWithNull	FAIL (TDigest diff)	PASS (per-engine value)
testStatsPercentileBySpan	FAIL (500 / split rule + TDigest diff)	PASS (per-engine value, paired with #21584 commit 5)
testDisableLegacyPreferred	FAIL	SKIPPED (semantic divergence pending team decision)
testStatsBySpanTimeWithNullBucket	FAIL (multi-unit span)	FAIL (intentionally — out of scope for this PR)
Overall	46 / 63 pass, 17 fail	52 / 63 pass, 10 fail, 1 skipped

Default behavior unchanged

With the flag unset, the cluster's persistent settings are untouched, indices stay Lucene-backed, and every IT runs through the existing V2 / Calcite path. CI that doesn't opt in sees zero behavior change.

Remaining 10 failures on CalciteStatsCommandIT (analytics path)

Bucket	Count	Notes
Sort-on-measure ties	6	Fixed by opensearch-project/sql#5435 (tie-tolerant matcher). When both PRs merge: combined 58 / 63 + 1 skipped + 4 fail.
testStatsTimeSpan datetime output-cast format	1	DataFusion's CAST(timestamp AS VARCHAR) emits ISO 8601 with T separator while PPL's Calcite path emits space-separated ANSI SQL format. Tracked at opensearch-project/sql#5420.
testStatsBySpanTimeWithNullBucket multi-unit time span	1	span(@timestamp, 12h) falls through SpanAdapter (only interval=1 is rewritten to DATE_TRUNC); needs DataFusion's date_bin. Left failing on the analytics path so the gap is visible.
LogicalJoin planner gap	2	Out of scope — analytics-engine planner doesn't support joins.

Test plan

• ./gradlew :integ-test:compileTestJava clean.
• Spotless clean.
• CalciteStatsCommandIT against the analytics-engine cluster with only -Dtests.analytics.parquet_indices=true: 46 → 52 passes + 1 documented skip.
• CI on this PR — default branches untouched, no regression expected on V2 / Calcite paths.

[dai-chen]

np: make both these index settings and refresh config into a test index config in single place. We have this problem in os-spark repo which has aoss conditional check and configs in many places.

[dai-chen]

Just thinking does all these assertion changes means semantic/behavior change? Or we can simply fix the assertion in this way.