fix: admin-adjustable max columns with a clear overflow message

[Ma77Ball]

What changes were proposed in this PR?

• Admin-adjustable max columns. Adds csv_parser_max_columns to Admin → Settings → Result Panel. The CSV scan operator reads this from site_settings at runtime.
• Readable overflow error. When a CSV row exceeds the limit, the operator now throws RuntimeException("Max columns of N exceeded.") Instead of letting Univocity's raw stack trace through. Detection looks at the cause
  (ArrayIndexOutOfBoundsException) and parses the index from both the Java 8 and Java 9+ message formats — needed because Univocity's own hint string is silently dropped on Java 9+.
• Refactor. Extracts the repeated "read int/long from site_settings with fallback" pattern into common/dao/SiteSettings, replacing inline jOOQ in CSVScanSourceOpExec and DatasetResource.

Any related issues, documentation, discussions?

closes #4589
Related #3825

How was this PR tested?

• New CSVScanSourceOpExecSpec: happy path, end-of-input, overflow translation against a real parser, and isColumnOverflow cases for both AIOOBE message formats.
• Expanded admin-settings.component.spec.ts for load/save/reset of the max columns setting.
• sbt Test/compile clean across DAO, ConfigService, WorkflowOperator, FileService. Frontend specs need Node ≥ 20.19 to run.
• Manual: lowered the limit, ran a workflow on a CSV that exceeded it, confirmed the result panel shows Max columns of N exceeded.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with: Claude Opus 4.7

[aglinxinyuan]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

[Yicong-Huang]

#3825 is already closed?

[chenlica]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

We want this parameter to be adjustable by the admins. I am OK with fixing it.

[chenlica]

@kunwp1 @SarahAsad23 Feel free to chime in based on a related use case with a collaborator.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 68.85246% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.71%. Comparing base (16d3c6a) to head (cb2aa5c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...mponent/admin/settings/admin-settings.component.ts	50.00%	8 Missing ⚠️
...operator/source/scan/csv/CSVScanSourceOpExec.scala	64.70%	4 Missing and 2 partials ⚠️
...ain/scala/org/apache/texera/dao/SiteSettings.scala	33.33%	4 Missing ⚠️
...onent/admin/settings/admin-settings.component.html	94.11%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #4669      +/-   ##
============================================
+ Coverage     42.69%   42.71%   +0.02%     
+ Complexity     2186     2183       -3     
============================================
  Files          1031     1032       +1     
  Lines         38111    38161      +50     
  Branches       4004     4006       +2     
============================================
+ Hits          16270    16301      +31     
- Misses        20825    20841      +16     
- Partials       1016     1019       +3     

Flag	Coverage Δ		*Carryforward flag
access-control-service	39.53% <ø> (ø)
agent-service	33.72% <ø> (ø)		Carriedforward from 083a091
amber	43.18% <61.53%> (+0.01%)	⬆️
computing-unit-managing-service	0.00% <ø> (ø)
config-service	0.00% <ø> (ø)
file-service	32.18% <100.00%> (-1.06%)	⬇️
frontend	33.09% <72.72%> (+0.08%)	⬆️
python	88.90% <ø> (+0.02%)	⬆️	Carriedforward from 083a091
workflow-compiling-service	47.72% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[aglinxinyuan]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

We want this parameter to be adjustable by the admins. I am OK with fixing it.

Let's make it a large fixed number, unless there is a disadvantage for that.

[Ma77Ball]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

We want this parameter to be adjustable by the admins. I am OK with fixing it.

Let's make it a large fixed number, unless there is a disadvantage for that.

The disadvantage is that univocity allocates memory per parser based on that number (10,000 is negligible, but something like 1,000,000 is not when you want to scale the platform).

[Ma77Ball]

#3825 is already closed?

That PR was closed after we decided in a meeting to keep it static, as @aglinxinyuan suggested (it only fixes the display of CSVs in the result panel with more columns). The problem is that users are hitting the 512 limit and need it increased.

[aglinxinyuan]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

We want this parameter to be adjustable by the admins. I am OK with fixing it.

Let's make it a large fixed number, unless there is a disadvantage for that.

The disadvantage is that univocity allocates memory per parser based on that number (10,000 is negligible, but something like 1,000,000 is not when you want to scale the platform).

This makes sense to me now. Why do we also need batch size adjustable for result table? Can we just pick a fixed number?

[Ma77Ball]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

We want this parameter to be adjustable by the admins. I am OK with fixing it.

Let's make it a large fixed number, unless there is a disadvantage for that.

The disadvantage is that univocity allocates memory per parser based on that number (10,000 is negligible, but something like 1,000,000 is not when you want to scale the platform).

This makes sense to me now. Why do we also need batch size adjustable for result table? Can we just pick a fixed number?

That setting I added because the result panel settings menu seems a little empty, and I thought it might be a nice feature for users. I can remove it if it should be in a different PR or if it's not needed.

[aglinxinyuan]

Why do we want to make this Admin-adjustable? I don't feel this makes much sense.

It should be either fixed at a large number or user-adjustable.

We want this parameter to be adjustable by the admins. I am OK with fixing it.

Let's make it a large fixed number, unless there is a disadvantage for that.

The disadvantage is that univocity allocates memory per parser based on that number (10,000 is negligible, but something like 1,000,000 is not when you want to scale the platform).

This makes sense to me now. Why do we also need batch size adjustable for result table? Can we just pick a fixed number?

That setting I added because the result panel settings menu seems a little empty, and I thought it might be a nice feature for users. I can remove it if it should be in a different PR or if it's not needed.

Let's don't over complicate the settings.

[chenlica]

Agreed. Let's take a simple solution.

[Ma77Ball]

@aglinxinyuan, please review again. I removed the other column-per-result-panel setting.

[Ma77Ball]

@aglinxinyuan please review

[aglinxinyuan]

PR description still mentions result_table_columns_per_batch. Please update.

[aglinxinyuan]

Left a few comments.

[aglinxinyuan]

All four items addressed, thanks. Two small nits inline, plus PR description still says "the default lives in default.conf" (no longer true after the csv block was removed) and lists ConfigResource in the refactor scope (no longer touched). Please update.

[aglinxinyuan]

Please fix the frontend format.