### What changes were proposed in this PR?  - **Admin-adjustable max columns.** Adds `csv_parser_max_columns` to Admin → Settings → Result Panel. The CSV scan operator reads this from `site_settings` at runtime. - **Readable overflow error.** When a CSV row exceeds the limit, the operator now throws `RuntimeException("Max columns of N exceeded.")` Instead of letting Univocity's raw stack trace through. Detection looks at the cause (`ArrayIndexOutOfBoundsException`) and parses the index from both the Java 8 and Java 9+ message formats — needed because Univocity's own hint string is silently dropped on Java 9+. - **Refactor.** Extracts the repeated "read int/long from `site_settings` with fallback" pattern into `common/dao/SiteSettings`, replacing inline jOOQ in `CSVScanSourceOpExec` and `DatasetResource`. ### Any related issues, documentation, discussions?  closes #4589 Related #3825 ### How was this PR tested?  - New `CSVScanSourceOpExecSpec`: happy path, end-of-input, overflow translation against a real parser, and `isColumnOverflow` cases for both AIOOBE message formats. - Expanded `admin-settings.component.spec.ts` for load/save/reset of the max columns setting. - `sbt Test/compile` clean across DAO, ConfigService, WorkflowOperator, FileService. Frontend specs need Node ≥ 20.19 to run. - Manual: lowered the limit, ran a workflow on a CSV that exceeded it, confirmed the result panel shows `Max columns of N exceeded.` ### Was this PR authored or co-authored using generative AI tooling?  Co-authored with: Claude Opus 4.7 --------- Co-authored-by: Chen Li <chenli@gmail.com> Co-authored-by: Xinyuan Lin <xinyual3@uci.edu>

CSV File Scan cannot read a csv with more than 512 columns #4589

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions