UN-3403 [FEAT] Agentic table extractor plugin with multi-agent LLM-powered table extraction#1914
UN-3403 [FEAT] Agentic table extractor plugin with multi-agent LLM-powered table extraction#1914harini-venkataraman wants to merge 105 commits intomainfrom
Conversation
Conflicts resolved: - docker-compose.yaml: Use main's dedicated dashboard_metric_events queue for worker-metrics - PromptCard.jsx: Keep tool_id matching condition from our async socket feature - PromptRun.jsx: Merge useEffect import from main with our branch - ToolIde.jsx: Keep fire-and-forget socket approach (spinner waits for socket event) - SocketMessages.js: Keep both session-store and socket-custom-tool imports + updateCusToolMessages dep - SocketContext.js: Keep simpler path-based socket connection approach - usePromptRun.js: Keep Celery fire-and-forget with socket delivery over polling - setupProxy.js: Accept main's deletion (migrated to Vite)
for more information, see https://pre-commit.ci
… into feat/execution-backend
for more information, see https://pre-commit.ci
… into feat/execution-backend
- Route _handle_structure_pipeline to _handle_single_pass_extraction when is_single_pass=True (was always calling _handle_answer_prompt) - Delegate _handle_single_pass_extraction to cloud plugin via ExecutorRegistry, falling back to _handle_answer_prompt if plugin not installed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a new complete_vision() method alongside existing complete() that accepts pre-built multimodal messages (text + image_url) in OpenAI-style format. LiteLLM auto-translates for Anthropic/Bedrock/Vertex providers. This enables the agentic table extractor plugin to send page images alongside text prompts for VLM-based table detection and extraction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- PromptCardItems loads AgenticTableChecklist plugin and owns the isAgenticTableReady state, rendering the checklist above the prompt text area and delegating the settings gear visibility to the plugin. - Header and PromptOutput disable their Run buttons when isAgenticTableReady is false (default true for non-agentic types). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts: # backend/api_v2/deployment_helper.py # backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py # backend/prompt_studio/prompt_studio_core_v2/views.py # docker/docker-compose.yaml # docker/sample.compose.override.yaml # frontend/src/components/custom-tools/prompt-card/DisplayPromptResult.jsx # frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx # frontend/src/components/custom-tools/prompt-card/PromptOutput.jsx # frontend/src/components/custom-tools/tools-main/ToolsMain.jsx # frontend/src/hooks/usePromptRun.js # frontend/src/hooks/usePromptStudioSocket.js # unstract/sdk1/src/unstract/sdk1/execution/dispatcher.py # unstract/sdk1/src/unstract/sdk1/execution/result.py # unstract/sdk1/src/unstract/sdk1/llm.py # unstract/sdk1/tests/test_execution.py # uv.lock # workers/executor/README.md # workers/executor/executors/index.py # workers/executor/executors/legacy_executor.py # workers/executor/executors/retrievers/base_retriever.py # workers/executor/executors/retrievers/fusion.py # workers/executor/executors/retrievers/keyword_table.py # workers/executor/executors/retrievers/router.py # workers/executor/executors/retrievers/subquestion.py # workers/executor/tasks.py # workers/executor/worker.py # workers/file_processing/structure_tool_task.py # workers/run-worker-docker.sh # workers/shared/enums/task_enums.py # workers/shared/enums/worker_enums_base.py # workers/shared/infrastructure/config/registry.py # workers/tests/test_answer_prompt.py # workers/tests/test_retrieval.py
ToolStudioPrompt uses prompt_id as its primary key, not id.
Count("id") causes FieldError on the list endpoint (500).
Co-authored-by: Chandrasekharan M <chandrasekharan@zipstack.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The cloud build adds "agentic_table" to the prompt enforce_type dropdown, but the OSS ToolStudioPrompt model rejected it as an invalid choice. Add AGENTIC_TABLE to EnforceType and ship a matching migration so the value can be persisted. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The single-prompt run flow had no branch for prompts with enforce_type=agentic_table, so clicking Run silently fell through to the legacy prompt-service path and never invoked the agentic_table executor. Adds an AGENTIC_TABLE constant to TSPKeys, includes it in the OperationNotSupported guard, and dispatches to PayloadModifier.execute_agentic_table when the plugin is available so the result still flows through _handle_response. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ExecutionDispatcher derives the queue name from the executor name
(celery_executor_{name}), so dispatches to the agentic_table executor
land on celery_executor_agentic_table. The local docker-compose default
only listed celery_executor_legacy and celery_executor_agentic, so no
worker consumed the new queue and dispatch hung for the full 1-hour
result timeout. Adds the missing queue to the docker-compose default.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The IDE Run button was building a legacy answer_prompt payload for agentic_table prompts, so the agentic table executor was never invoked. Branch fetch_response on enforce_type so agentic_table prompts are built via the cloud payload_modifier plugin and dispatched directly to celery_executor_agentic_table. Add the enforce_type to the OSS dropdown choices and the JSON-dump set in OutputManagerHelper so the persisted output is parseable by the FE table renderer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The agentic_table executor returns {"output": {"tables": [...],
"page_count": ..., "headers": [...], ...}}, but
OutputManagerHelper.handle_prompt_output_update reads
outputs[prompt.prompt_key] when persisting prompt output. Without a
reshape the table list never lands under the prompt key and the FE
sees an empty result.
When cb_kwargs carries is_agentic_table=True and prompt_key (set by
the cloud build_agentic_table_payload), reshape outputs to
{prompt_key: tables} before calling update_prompt_output. The
executor itself also shapes its envelope, so this is a defensive
double-keying that keeps the legacy answer_prompt path untouched.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary by CodeRabbitRelease Notes
WalkthroughThe pull request introduces a new "agentic_table" response type throughout the prompt studio system. Changes include adding the new constant across backend modules, updating execution pipelines to dispatch agentic table prompts through specialized handlers, modifying output management and registry export logic, adding frontend UI readiness states, extending worker executors with agentic table handling, and updating infrastructure configuration to support new execution queues. Changes
Sequence Diagram(s)sequenceDiagram
actor User
participant Frontend as Frontend UI
participant Views as Backend Views
participant Helper as Backend Helper
participant Dispatcher as Task Dispatcher
participant Executor as Worker Executor
participant Callback as IDE Callback
participant Output as Output Manager
User->>Frontend: Trigger agentic table prompt run
Frontend->>Views: POST fetch_response (enforce_type=AGENTIC_TABLE)
Views->>Views: Load payload_modifier plugin
Views->>Views: Build agentic_table_payload via modifier
Views->>Views: Generate executor_task_id and dispatch_time
Views->>Dispatcher: dispatch_with_callback(task, callback_config)
Dispatcher-->>Views: Return task_id
Views-->>Frontend: HTTP 202 ACCEPTED {task_id}
Dispatcher->>Executor: Execute agentic table extraction
Executor->>Executor: Partition outputs (agentic vs regular)
Executor->>Executor: Dispatch agentic prompts individually
Executor-->>Dispatcher: Return structured results
Dispatcher->>Callback: ide_prompt_complete(results)
Callback->>Callback: Reshape output {prompt_key: tables}
Callback->>Output: update_prompt_output(reshaped_output)
Output->>Output: JSON serialize for agentic_table type
Output-->>Callback: Confirm storage
Callback-->>Frontend: Notify completion
Frontend-->>User: Display results
Estimated code review effort🎯 4 (Complex) | ⏱️ ~70 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: harini-venkataraman <115449948+harini-venkataraman@users.noreply.github.com>
for more information, see https://pre-commit.ci
|
|
| Filename | Overview |
|---|---|
| backend/prompt_studio/prompt_studio_core_v2/views.py | Adds an agentic table dispatch path in fetch_response; logic is consistent with the existing regular path but is_agentic_table flag in cb_kwargs relies on the closed-source plugin to set it |
| unstract/sdk1/src/unstract/sdk1/llm.py | Adds complete_vision method; sets litellm.drop_params = True as a global side-effect that persists for the process and affects all subsequent complete() calls |
| workers/file_processing/structure_tool_task.py | Partitions prompts into agentic vs regular and dispatches them separately; mixed-case merge logic (setdefault("output", {}).update(agentic_results)) is correct |
| workers/ide_callback/tasks.py | Output reshape for agentic executor response depends on is_agentic_table flag being set by the closed-source plugin; if omitted, persisted output will be empty |
| workers/executor/executors/legacy_executor.py | Adds defensive guard for agentic_table prompts; also refactors email "NA" handling via _convert_scalar_answer (intentional behavioral change confirmed by updated tests) |
| frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx | Adds AgenticTableChecklist plugin import and isAgenticTableReady gate; removes enforceType === TABLE outer guard on TableExtractionSettingsBtn, which now renders for all enforce types |
| backend/prompt_studio/prompt_studio_v2/migrations/0014_alter_toolstudioprompt_enforce_type.py | Adds agentic_table choice to the enforce_type field; migration is additive and safe |
| frontend/src/hooks/usePromptRun.js | Increases socket timeout from 5 min to 16 min to match the server-side 900 s LLM adapter timeout; well-commented and correct |
| workers/tests/test_answer_prompt.py | Updates tests to reflect intentional behavioral change: email "NA" answers now become None instead of being preserved as-is |
Sequence Diagram
sequenceDiagram
participant FE as Frontend (PromptCardItems)
participant BE as Backend (views.py fetch_response)
participant PM as PayloadModifier Plugin
participant EX as Celery: agentic_table executor
participant CB as Celery: ide_callback
FE->>BE: POST fetch_response (enforce_type=agentic_table)
BE->>PM: build_agentic_table_payload(...)
PM-->>BE: context, cb_kwargs (incl. is_agentic_table, prompt_key)
BE->>EX: dispatch_with_callback(context, task_id)
BE-->>FE: 202 Accepted {task_id, run_id}
EX-->>CB: on_success → ide_prompt_complete(result_dict, cb_kwargs)
CB->>CB: reshape output {tables:[...]} → {prompt_key: tables}
CB->>BE: update_prompt_output(outputs, prompt_ids, ...)
FE-->>FE: WebSocket notification → render output
Comments Outside Diff (2)
-
unstract/sdk1/src/unstract/sdk1/llm.py, line 517 (link)Global
drop_paramsmutation affects all litellm callslitellm.drop_params = Trueis a module-level global that persists for the entire process lifetime. After the firstcomplete_vision()call, every subsequentlitellm.completion()fromcomplete()will also silently drop unknown parameters, even thoughcomplete()was never designed with that expectation. This can hide parameter-mismatch errors that would otherwise be surfaced. Passdrop_params=Trueper-call instead:And remove the
litellm.drop_params = Trueline at line 517.Prompt To Fix With AI
This is a comment left during a code review. Path: unstract/sdk1/src/unstract/sdk1/llm.py Line: 517 Comment: **Global `drop_params` mutation affects all litellm calls** `litellm.drop_params = True` is a module-level global that persists for the entire process lifetime. After the first `complete_vision()` call, every subsequent `litellm.completion()` from `complete()` will also silently drop unknown parameters, even though `complete()` was never designed with that expectation. This can hide parameter-mismatch errors that would otherwise be surfaced. Pass `drop_params=True` per-call instead: And remove the `litellm.drop_params = True` line at line 517. How can I resolve this? If you propose a fix, please make it concise.
-
frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx, line 397-398 (link)TableExtractionSettingsBtnnow renders for every enforce typeThe
enforceType === TABLEguard was removed, soTableExtractionSettingsBtnis rendered for all enforce types (text, json, boolean, date, …), not justtableandagentic_table. Because the component receivesenforceTypeas a prop it can self-gate, but if the plugin implementation does not checkenforceTypeinternally, users will see the table-settings gear on every prompt card regardless of type. Was the intent to show it only fortableandagentic_table, and has the plugin been updated to do that check? Does theTableExtractionSettingsBtnplugin component internally checkenforceType(e.g., only render fortable/agentic_table), or should the outer guard still limit rendering to those two types?Prompt To Fix With AI
This is a comment left during a code review. Path: frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx Line: 397-398 Comment: **`TableExtractionSettingsBtn` now renders for every enforce type** The `enforceType === TABLE` guard was removed, so `TableExtractionSettingsBtn` is rendered for all enforce types (text, json, boolean, date, …), not just `table` and `agentic_table`. Because the component receives `enforceType` as a prop it can self-gate, but if the plugin implementation does not check `enforceType` internally, users will see the table-settings gear on every prompt card regardless of type. Was the intent to show it only for `table` and `agentic_table`, and has the plugin been updated to do that check? Does the `TableExtractionSettingsBtn` plugin component internally check `enforceType` (e.g., only render for `table` / `agentic_table`), or should the outer guard still limit rendering to those two types? How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: unstract/sdk1/src/unstract/sdk1/llm.py
Line: 517
Comment:
**Global `drop_params` mutation affects all litellm calls**
`litellm.drop_params = True` is a module-level global that persists for the entire process lifetime. After the first `complete_vision()` call, every subsequent `litellm.completion()` from `complete()` will also silently drop unknown parameters, even though `complete()` was never designed with that expectation. This can hide parameter-mismatch errors that would otherwise be surfaced. Pass `drop_params=True` per-call instead:
```suggestion
response: dict[str, object] = litellm.completion(
messages=messages,
drop_params=True,
**completion_kwargs,
)
```
And remove the `litellm.drop_params = True` line at line 517.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx
Line: 397-398
Comment:
**`TableExtractionSettingsBtn` now renders for every enforce type**
The `enforceType === TABLE` guard was removed, so `TableExtractionSettingsBtn` is rendered for all enforce types (text, json, boolean, date, …), not just `table` and `agentic_table`. Because the component receives `enforceType` as a prop it can self-gate, but if the plugin implementation does not check `enforceType` internally, users will see the table-settings gear on every prompt card regardless of type. Was the intent to show it only for `table` and `agentic_table`, and has the plugin been updated to do that check?
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: workers/ide_callback/tasks.py
Line: 399-403
Comment:
**`is_agentic_table` flag not visibly set in `cb_kwargs`**
The reshape at line 399–403 only fires when `cb.get("is_agentic_table")` is truthy. That flag must be injected into `cb_kwargs` by `modifier.build_agentic_table_payload(...)` in the cloud plugin. If the plugin omits it (or changes its key name), the agentic executor's `{"tables": [...]}` payload will be forwarded verbatim to `update_prompt_output`, which will find no value under `prompt_key` and persist an empty result silently. Consider adding an explicit `cb_kwargs["is_agentic_table"] = True` in `views.py` after the `modifier.build_agentic_table_payload` call — it keeps the contract visible in the open-source code and doesn't depend on the plugin remembering to set it.
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "Fixing syntax issues" | Re-trigger Greptile
| if cb.get("is_agentic_table"): | ||
| prompt_key = cb.get("prompt_key", "") | ||
| if prompt_key: | ||
| tables = outputs.get("tables", []) if isinstance(outputs, dict) else [] | ||
| outputs = {prompt_key: tables} |
There was a problem hiding this comment.
is_agentic_table flag not visibly set in cb_kwargs
The reshape at line 399–403 only fires when cb.get("is_agentic_table") is truthy. That flag must be injected into cb_kwargs by modifier.build_agentic_table_payload(...) in the cloud plugin. If the plugin omits it (or changes its key name), the agentic executor's {"tables": [...]} payload will be forwarded verbatim to update_prompt_output, which will find no value under prompt_key and persist an empty result silently. Consider adding an explicit cb_kwargs["is_agentic_table"] = True in views.py after the modifier.build_agentic_table_payload call — it keeps the contract visible in the open-source code and doesn't depend on the plugin remembering to set it.
Prompt To Fix With AI
This is a comment left during a code review.
Path: workers/ide_callback/tasks.py
Line: 399-403
Comment:
**`is_agentic_table` flag not visibly set in `cb_kwargs`**
The reshape at line 399–403 only fires when `cb.get("is_agentic_table")` is truthy. That flag must be injected into `cb_kwargs` by `modifier.build_agentic_table_payload(...)` in the cloud plugin. If the plugin omits it (or changes its key name), the agentic executor's `{"tables": [...]}` payload will be forwarded verbatim to `update_prompt_output`, which will find no value under `prompt_key` and persist an empty result silently. Consider adding an explicit `cb_kwargs["is_agentic_table"] = True` in `views.py` after the `modifier.build_agentic_table_payload` call — it keeps the contract visible in the open-source code and doesn't depend on the plugin remembering to set it.
How can I resolve this? If you propose a fix, please make it concise.
Frontend Lint Report (Biome)✅ All checks passed! No linting or formatting issues found. |
Test ResultsSummary
Runner Tests - Full Report
SDK1 Tests - Full Report
|
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx (1)
300-306:⚠️ Potential issue | 🟡 MinorKeep the table-settings button behind an enforce-type gate.
This now renders the settings entry for every prompt as soon as the plugin is installed, including text/number/email prompts. That is confusing at best, and it makes it easier to save table-specific config on incompatible prompt types.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx` around lines 300 - 306, The TableExtractionSettingsBtn is being rendered for all prompts; guard its render with the enforce-type check so the settings only appear for table-enforced prompts. Update the conditional around TableExtractionSettingsBtn in PromptCardItems.jsx (the JSX block that currently uses TableExtractionSettingsBtn, promptDetails?.prompt_id, enforceType, setAllTableSettings) to require a table-specific enforceType (e.g., enforceType === 'table' or enforceType?.includes('table')) in addition to TableExtractionSettingsBtn before rendering the component, so incompatible prompt types won’t show the table settings.
🧹 Nitpick comments (5)
unstract/sdk1/src/unstract/sdk1/llm.py (1)
390-390: Avoid per-call global mutation oflitellm.drop_params.Line 390 reassigns a module-global already initialized at Line 33; this contradicts the module-level intent to avoid repeated global mutation per request.
♻️ Proposed fix
- litellm.drop_params = True🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@unstract/sdk1/src/unstract/sdk1/llm.py` at line 390, Remove the per-call reassignment of the module-global litellm.drop_params (the assignment at the shown call site) and instead either set the desired value once at module initialization where litellm is imported/initialized (the earlier initialization around line 33) or avoid mutating the global by using a local variable (e.g., drop_params) and pass that into the litellm API calls; in short, delete the litellm.drop_params = True line and either consolidate the flag into module-level setup or thread a local parameter through the functions that invoke litellm.frontend/src/hooks/usePromptRun.js (1)
19-23: Prefer a config-driven timeout instead of a fixed 16-minute constant.Line 23 can silently drift from server adapter settings across environments. Consider sourcing this value from backend-exposed config (with buffer applied client-side) to avoid premature UI timeout regressions after infra changes.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@frontend/src/hooks/usePromptRun.js` around lines 19 - 23, The hardcoded SOCKET_TIMEOUT_MS constant in usePromptRun.js can drift from server adapter settings; change it to derive the timeout from a backend-exposed config value (e.g., an API response or injected runtime config) and apply the client-side buffer (e.g., subtract or add the intended 1 minute) when computing SOCKET_TIMEOUT_MS; implement a safe fallback to the current 16-minute value if the backend config is unavailable, and update any functions using SOCKET_TIMEOUT_MS so they reference the computed/config-driven value instead of the hardcoded constant.docker/docker-compose.yaml (1)
532-532: Good queue addition—mirror this default in all deployment targets.Line 532 is correct for local/dev, but please ensure Helm/chart and runtime env defaults include
celery_executor_agentic_tableas well, or agentic-table jobs can remain unconsumed in some environments.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docker/docker-compose.yaml` at line 532, The docker-compose default for CELERY_QUEUES_EXECUTOR was extended to include celery_executor_agentic_table but other deployment targets are missing it; update all runtime/defaults to match by adding celery_executor_agentic_table to the CELERY_QUEUES_EXECUTOR default in Helm values (values.yaml), chart Deployment/StatefulSet env entries (templates/* where CELERY_QUEUES_EXECUTOR is set), and any CI/runtime environment variable configs (e.g., container env vars, systemd or cloud run settings) so every environment uses "celery_executor_legacy,celery_executor_agentic,celery_executor_agentic_table" as the default queue list.backend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.py (1)
173-179: Use centralized enforce-type constants here to avoid string drift.The new
agentic_tablebranch is correct, but this block is still string-literal based. Switching to shared constants will prevent future typo/divergence bugs.♻️ Suggested refactor
+from prompt_studio.prompt_studio_core_v2.constants import ( + ToolStudioPromptKeys as TSPKeys, +) ... - if prompt.enforce_type in { - "json", - "table", - "record", - "line-item", - "agentic_table", - }: + if prompt.enforce_type in { + TSPKeys.JSON, + TSPKeys.TABLE, + TSPKeys.RECORD, + TSPKeys.LINE_ITEM, + TSPKeys.AGENTIC_TABLE, + }: output = json.dumps(output)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.py` around lines 173 - 179, Replace the literal string set check on prompt.enforce_type with the centralized enforce-type constants: import and use the shared constants for JSON/TABLE/RECORD/LINE_ITEM/AGENTIC_TABLE (e.g., ENFORCE_TYPE_JSON, ENFORCE_TYPE_TABLE, ENFORCE_TYPE_RECORD, ENFORCE_TYPE_LINE_ITEM, ENFORCE_TYPE_AGENTIC_TABLE) from the module that defines enforce-type values (the centralized constants module in prompt_studio), and change the condition in output_manager_helper.py (the prompt.enforce_type check) to use those constants instead of the string literals to avoid string drift.workers/file_processing/structure_tool_task.py (1)
402-442: Consider defensive access forllmandnamekeys to provide clearer error messages.Lines 414 and 442 use direct key access (
at_output["llm"],at_output[_SK.NAME]) which will raiseKeyErrorwith a generic traceback if missing. Since the validation block (lines 302-313) only checksagentic_table_settings, these fields aren't validated beforehand.If the export process guarantees these keys, this is acceptable. Otherwise, wrapping in explicit checks would produce actionable error messages matching the style at lines 305-313.
🔧 Optional: Add explicit validation for required output keys
for at_output in agentic_table_outputs: at_settings = at_output.get("agentic_table_settings") or {} + if not at_output.get(_SK.NAME) or not at_output.get("llm"): + return ExecutionResult.failure( + error=( + f"Agentic table output is missing required 'name' or 'llm' key. " + f"Re-export the tool from Prompt Studio." + ) + ).to_dict() if not at_settings.get("target_table") or not at_settings.get("json_structure"):🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@workers/file_processing/structure_tool_task.py` around lines 402 - 442, The loop over agentic_table_outputs accesses at_output["llm"] and at_output[_SK.NAME] directly which can raise KeyError; add defensive validation before using them (e.g., confirm required keys in each at_output or use at_output.get(...) and raise/return a clear error) so failures mirror the earlier validation style for agentic_table_settings; specifically check each entry in agentic_table_outputs for "llm" and _SK.NAME (or provide sensible defaults) before building agentic_params and before assigning agentic_results[...], and if missing return a structured error result (similar to other validation paths) rather than letting a KeyError bubble from dispatcher.dispatch/ExecutionContext.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py`:
- Around line 1574-1590: The single-pass prompt filter must also exclude
agentic-table prompts to prevent them from being bundled into legacy single-pass
execution; update the single-pass filter logic (the code that currently excludes
only TSPKeys.TABLE and TSPKeys.RECORD) to additionally exclude
TSPKeys.AGENTIC_TABLE by checking prompt_instance.enforce_type ==
TSPKeys.AGENTIC_TABLE (same symbol used in the single-prompt branch) so
agentic-table prompts follow the payload_modifier_plugin path and do not end up
in legacy_executor silent skips.
In `@frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx`:
- Around line 94-95: The `isAgenticTableReady` state is being used globally
causing non-agentic prompts to be blocked; scope this readiness to only
agentic_table prompts by initializing and updating `isAgenticTableReady` based
on `promptDetails?.prompt_type === 'agentic_table'` (use the
`promptDetails`/`promptId` context) and reset it to true or undefined when
`promptDetails.prompt_type` changes away from 'agentic_table'; update the places
that read this flag (components/functions `Header`, `PromptOutput`, and any
setters in `PromptCardItems.jsx` such as the `setIsAgenticTableReady` usage) so
they only disable run buttons when the current prompt is of type 'agentic_table'
and the readiness flag is false.
In `@workers/executor/executors/legacy_executor.py`:
- Around line 1873-1885: The current guard in the legacy executor silently
returns when output_type == "agentic_table", leaving
structured_output[prompt_name] unset; change this to raise an explicit exception
instead so the run fails visibly: in the same block that checks output_type ==
"agentic_table" (using variables output_type and prompt_name and logger),
replace the silent return with raising a clear exception (e.g., RuntimeError or
ValueError) that includes prompt_name and a message stating the prompt was
misrouted and should have been dispatched to the agentic_table executor; keep
the logger.warning call if you want a log entry before raising so the error is
recorded.
In `@workers/ide_callback/tasks.py`:
- Around line 395-403: The current branch for cb.get("is_agentic_table")
incorrectly replaces the full executor payload with only outputs["tables"],
discarding fields like page_count and headers; instead, preserve the entire
payload by nesting it under the prompt key before calling
update_prompt_output(): when cb.get("is_agentic_table") and prompt_key is set,
set outputs = {prompt_key: outputs} (if outputs is already a dict, wrap that
dict; if it isn't, wrap the original value as-is) so update_prompt_output()
receives the complete agentic-table payload (reference symbols: cb, prompt_key,
outputs, update_prompt_output, is_agentic_table).
---
Outside diff comments:
In `@frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx`:
- Around line 300-306: The TableExtractionSettingsBtn is being rendered for all
prompts; guard its render with the enforce-type check so the settings only
appear for table-enforced prompts. Update the conditional around
TableExtractionSettingsBtn in PromptCardItems.jsx (the JSX block that currently
uses TableExtractionSettingsBtn, promptDetails?.prompt_id, enforceType,
setAllTableSettings) to require a table-specific enforceType (e.g., enforceType
=== 'table' or enforceType?.includes('table')) in addition to
TableExtractionSettingsBtn before rendering the component, so incompatible
prompt types won’t show the table settings.
---
Nitpick comments:
In
`@backend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.py`:
- Around line 173-179: Replace the literal string set check on
prompt.enforce_type with the centralized enforce-type constants: import and use
the shared constants for JSON/TABLE/RECORD/LINE_ITEM/AGENTIC_TABLE (e.g.,
ENFORCE_TYPE_JSON, ENFORCE_TYPE_TABLE, ENFORCE_TYPE_RECORD,
ENFORCE_TYPE_LINE_ITEM, ENFORCE_TYPE_AGENTIC_TABLE) from the module that defines
enforce-type values (the centralized constants module in prompt_studio), and
change the condition in output_manager_helper.py (the prompt.enforce_type check)
to use those constants instead of the string literals to avoid string drift.
In `@docker/docker-compose.yaml`:
- Line 532: The docker-compose default for CELERY_QUEUES_EXECUTOR was extended
to include celery_executor_agentic_table but other deployment targets are
missing it; update all runtime/defaults to match by adding
celery_executor_agentic_table to the CELERY_QUEUES_EXECUTOR default in Helm
values (values.yaml), chart Deployment/StatefulSet env entries (templates/*
where CELERY_QUEUES_EXECUTOR is set), and any CI/runtime environment variable
configs (e.g., container env vars, systemd or cloud run settings) so every
environment uses
"celery_executor_legacy,celery_executor_agentic,celery_executor_agentic_table"
as the default queue list.
In `@frontend/src/hooks/usePromptRun.js`:
- Around line 19-23: The hardcoded SOCKET_TIMEOUT_MS constant in usePromptRun.js
can drift from server adapter settings; change it to derive the timeout from a
backend-exposed config value (e.g., an API response or injected runtime config)
and apply the client-side buffer (e.g., subtract or add the intended 1 minute)
when computing SOCKET_TIMEOUT_MS; implement a safe fallback to the current
16-minute value if the backend config is unavailable, and update any functions
using SOCKET_TIMEOUT_MS so they reference the computed/config-driven value
instead of the hardcoded constant.
In `@unstract/sdk1/src/unstract/sdk1/llm.py`:
- Line 390: Remove the per-call reassignment of the module-global
litellm.drop_params (the assignment at the shown call site) and instead either
set the desired value once at module initialization where litellm is
imported/initialized (the earlier initialization around line 33) or avoid
mutating the global by using a local variable (e.g., drop_params) and pass that
into the litellm API calls; in short, delete the litellm.drop_params = True line
and either consolidate the flag into module-level setup or thread a local
parameter through the functions that invoke litellm.
In `@workers/file_processing/structure_tool_task.py`:
- Around line 402-442: The loop over agentic_table_outputs accesses
at_output["llm"] and at_output[_SK.NAME] directly which can raise KeyError; add
defensive validation before using them (e.g., confirm required keys in each
at_output or use at_output.get(...) and raise/return a clear error) so failures
mirror the earlier validation style for agentic_table_settings; specifically
check each entry in agentic_table_outputs for "llm" and _SK.NAME (or provide
sensible defaults) before building agentic_params and before assigning
agentic_results[...], and if missing return a structured error result (similar
to other validation paths) rather than letting a KeyError bubble from
dispatcher.dispatch/ExecutionContext.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 745f3b34-3732-4f3c-9564-7de5c201cfcd
📒 Files selected for processing (21)
backend/prompt_studio/prompt_studio_core_v2/constants.pybackend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.pybackend/prompt_studio/prompt_studio_core_v2/static/select_choices.jsonbackend/prompt_studio/prompt_studio_core_v2/views.pybackend/prompt_studio/prompt_studio_output_manager_v2/output_manager_helper.pybackend/prompt_studio/prompt_studio_registry_v2/constants.pybackend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.pybackend/prompt_studio/prompt_studio_v2/migrations/0014_alter_toolstudioprompt_enforce_type.pybackend/prompt_studio/prompt_studio_v2/models.pydocker/docker-compose.yamlfrontend/src/components/custom-tools/prompt-card/Header.jsxfrontend/src/components/custom-tools/prompt-card/PromptCardItems.jsxfrontend/src/components/custom-tools/prompt-card/PromptOutput.jsxfrontend/src/hooks/usePromptRun.jsunstract/sdk1/src/unstract/sdk1/llm.pyworkers/executor/executors/legacy_executor.pyworkers/executor/executors/retrievers/fusion.pyworkers/executor/executors/retrievers/keyword_table.pyworkers/file_processing/structure_tool_task.pyworkers/ide_callback/tasks.pyworkers/tests/test_answer_prompt.py
| if ( | ||
| prompt_instance.enforce_type == TSPKeys.AGENTIC_TABLE | ||
| and payload_modifier_plugin | ||
| ): | ||
| modifier_service = payload_modifier_plugin["service_class"]() | ||
| response = modifier_service.execute_agentic_table( | ||
| tool_id=tool_id, | ||
| prompt_id=str(prompt_instance.prompt_id), | ||
| prompt_key=prompt_name, | ||
| prompt=prompt_instance.prompt, | ||
| doc_path=doc_path, | ||
| doc_name=doc_name, | ||
| org_id=org_id, | ||
| user_id=user_id, | ||
| run_id=run_id, | ||
| ) | ||
| else: |
There was a problem hiding this comment.
This only fixes the single-prompt path; single-pass can still misroute agentic_table.
AGENTIC_TABLE is special-cased here, but the same file’s single-pass prompt filter still excludes only TABLE and RECORD at Lines 1642-1646. That means agentic-table prompts can still be bundled into legacy single-pass execution and hit the silent skip in workers/executor/executors/legacy_executor.py.
Follow-up change needed in the single-pass filter
prompts = [
prompt
for prompt in prompts
if prompt.prompt_type != TSPKeys.NOTES
and prompt.active
and prompt.enforce_type != TSPKeys.TABLE
and prompt.enforce_type != TSPKeys.RECORD
+ and prompt.enforce_type != TSPKeys.AGENTIC_TABLE
]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/prompt_studio/prompt_studio_core_v2/prompt_studio_helper.py` around
lines 1574 - 1590, The single-pass prompt filter must also exclude agentic-table
prompts to prevent them from being bundled into legacy single-pass execution;
update the single-pass filter logic (the code that currently excludes only
TSPKeys.TABLE and TSPKeys.RECORD) to additionally exclude TSPKeys.AGENTIC_TABLE
by checking prompt_instance.enforce_type == TSPKeys.AGENTIC_TABLE (same symbol
used in the single-prompt branch) so agentic-table prompts follow the
payload_modifier_plugin path and do not end up in legacy_executor silent skips.
| const [isAgenticTableReady, setIsAgenticTableReady] = useState(true); | ||
| const promptId = promptDetails?.prompt_id; |
There was a problem hiding this comment.
Scope the readiness checklist to agentic_table prompts and reset the flag when leaving that type.
Right now the checklist can drive isAgenticTableReady for every prompt, and that flag is then fed into both Header and PromptOutput run-button disables. A false readiness emitted for a non-agentic prompt will block runs that should still be allowed.
Proposed fix
const [tableSettings, setTableSettings] = useState({});
const [isAgenticTableReady, setIsAgenticTableReady] = useState(true);
+
+ useEffect(() => {
+ if (enforceType !== "agentic_table") {
+ setIsAgenticTableReady(true);
+ }
+ }, [enforceType]);
...
- {AgenticTableChecklist && (
+ {enforceType === "agentic_table" && AgenticTableChecklist && (
<AgenticTableChecklist
promptId={promptDetails?.prompt_id}
promptText={promptText}
enforceType={enforceType}
onReadinessChange={setIsAgenticTableReady}
/>
)}Also applies to: 225-226, 236-243, 346-347
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@frontend/src/components/custom-tools/prompt-card/PromptCardItems.jsx` around
lines 94 - 95, The `isAgenticTableReady` state is being used globally causing
non-agentic prompts to be blocked; scope this readiness to only agentic_table
prompts by initializing and updating `isAgenticTableReady` based on
`promptDetails?.prompt_type === 'agentic_table'` (use the
`promptDetails`/`promptId` context) and reset it to true or undefined when
`promptDetails.prompt_type` changes away from 'agentic_table'; update the places
that read this flag (components/functions `Header`, `PromptOutput`, and any
setters in `PromptCardItems.jsx` such as the `setIsAgenticTableReady` usage) so
they only disable run buttons when the current prompt is of type 'agentic_table'
and the readiness flag is false.
| # Defensive guard: agentic_table prompts must be dispatched to | ||
| # the dedicated agentic_table executor by the worker (Layer 2 in | ||
| # workers/file_processing/structure_tool_task.py). If one ever | ||
| # reaches this method, the legacy fallthrough below would store | ||
| # the raw LLM completion as a string. Skip silently with a | ||
| # warning so the caller's existing entry (if any) survives. | ||
| if output_type == "agentic_table": | ||
| logger.warning( | ||
| "Skipping agentic_table prompt %s in legacy executor — " | ||
| "should have been dispatched to agentic_table executor", | ||
| prompt_name, | ||
| ) | ||
| return |
There was a problem hiding this comment.
Fail misrouted agentic_table prompts instead of silently dropping them.
This return leaves structured_output[prompt_name] unset, but the legacy answer flow still completes successfully. If an agentic-table prompt ever reaches this path, the run will look successful while persisting no value for that prompt.
Proposed fix
if output_type == "agentic_table":
- logger.warning(
- "Skipping agentic_table prompt %s in legacy executor — "
- "should have been dispatched to agentic_table executor",
- prompt_name,
- )
- return
+ raise LegacyExecutorError(
+ message=(
+ f"Prompt '{prompt_name}' with type 'agentic_table' "
+ "was routed to the legacy executor"
+ )
+ )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@workers/executor/executors/legacy_executor.py` around lines 1873 - 1885, The
current guard in the legacy executor silently returns when output_type ==
"agentic_table", leaving structured_output[prompt_name] unset; change this to
raise an explicit exception instead so the run fails visibly: in the same block
that checks output_type == "agentic_table" (using variables output_type and
prompt_name and logger), replace the silent return with raising a clear
exception (e.g., RuntimeError or ValueError) that includes prompt_name and a
message stating the prompt was misrouted and should have been dispatched to the
agentic_table executor; keep the logger.warning call if you want a log entry
before raising so the error is recorded.
| # Agentic table executor returns {"tables": [...], "page_count": ..., | ||
| # "headers": [...], ...}, but OutputManagerHelper expects | ||
| # outputs[prompt.prompt_key] to be the value for that prompt. Reshape | ||
| # so the table list lands under the prompt key. | ||
| if cb.get("is_agentic_table"): | ||
| prompt_key = cb.get("prompt_key", "") | ||
| if prompt_key: | ||
| tables = outputs.get("tables", []) if isinstance(outputs, dict) else [] | ||
| outputs = {prompt_key: tables} |
There was a problem hiding this comment.
Don't drop the rest of the agentic-table payload here.
The reshape keeps only outputs["tables"], so fields like page_count, headers, or any other executor output are discarded before update_prompt_output(). That makes the persisted result strictly less informative than what the executor returned.
Proposed fix
if cb.get("is_agentic_table"):
prompt_key = cb.get("prompt_key", "")
if prompt_key:
- tables = outputs.get("tables", []) if isinstance(outputs, dict) else []
- outputs = {prompt_key: tables}
+ payload = outputs if isinstance(outputs, dict) else {"tables": []}
+ outputs = {prompt_key: payload}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@workers/ide_callback/tasks.py` around lines 395 - 403, The current branch for
cb.get("is_agentic_table") incorrectly replaces the full executor payload with
only outputs["tables"], discarding fields like page_count and headers; instead,
preserve the entire payload by nesting it under the prompt key before calling
update_prompt_output(): when cb.get("is_agentic_table") and prompt_key is set,
set outputs = {prompt_key: outputs} (if outputs is already a dict, wrap that
dict; if it isn't, wrap the original value as-is) so update_prompt_output()
receives the complete agentic-table payload (reference symbols: cb, prompt_key,
outputs, update_prompt_output, is_agentic_table).
Additional Review Findings🔵 LOW:
|
jaseemjaskp
left a comment
There was a problem hiding this comment.
Additional Review Findings
Beyond what CodeRabbit and Greptile already flagged, here are additional issues found during a deeper review:
Critical
1. Silent incomplete export when payload_modifier plugin is missing
backend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.py — the new elif prompt.enforce_type == AGENTIC_TABLE block (around line 375)
When exporting an agentic_table prompt without the payload_modifier plugin available, the if payload_modifier_plugin: guard silently skips the call to export_agentic_table_settings. The export succeeds without agentic_table_settings, and the user only discovers this at document-processing time when structure_tool_task.py validation fails with "Re-export the tool from Prompt Studio."
This is a "fail later" anti-pattern — the failure should happen at export time:
elif prompt.enforce_type == PromptStudioRegistryKeys.AGENTIC_TABLE:
payload_modifier_plugin = get_plugin("payload_modifier")
if not payload_modifier_plugin:
raise OperationNotSupported(
"Agentic table export requires the payload_modifier plugin."
)
modifier_service = payload_modifier_plugin["service_class"]()
output = modifier_service.export_agentic_table_settings(...)Important
2. Missing prompt_key silently skips callback reshaping
workers/ide_callback/tasks.py — around line 397
When is_agentic_table=True but prompt_key is empty/missing, the if prompt_key: guard skips reshaping silently. The raw executor output ({"tables": [...], "page_count": ..., "headers": [...]}) gets persisted as-is with zero logging. Should log an error and fail explicitly rather than persisting malformed data.
3. No error handling around agentic table dispatch in views.py
backend/prompt_studio/prompt_studio_core_v2/views.py — lines ~512-562
The entire agentic table dispatch block (plugin instantiation, build_agentic_table_payload, dispatch_with_callback) runs without any try/except. Compare this to the existing indexing dispatch which wraps dispatch_with_callback in try/except with cleanup logic. If the cloud plugin's build_agentic_table_payload raises or the Celery broker is down, users get an opaque 500 with no actionable information.
4. Single agentic_table failure aborts ALL remaining prompts
workers/file_processing/structure_tool_task.py — around line 430
In the agentic_table dispatch loop, if any single prompt fails (if not at_result.success: return at_result.to_dict()), the function returns immediately — all subsequent agentic prompts AND the entire regular legacy pipeline are abandoned. For a tool with 10 prompts where only 1 is agentic_table, a failure in that one prompt produces zero output for all 10. At minimum, log the broader impact (how many prompts were abandoned).
5. 16-minute SOCKET_TIMEOUT_MS applies to ALL prompt types
frontend/src/hooks/usePromptRun.js — line 19
The timeout increase from 5→16 minutes is global. For regular text/number/email prompts that should complete in seconds, a stalled request now takes 16 minutes to surface a timeout error. Consider making the timeout type-aware (e.g. keep 5min for regular prompts, 16min for agentic_table).
Suggestions
6. Inaccurate comments referencing non-existent terminology
workers/executor/executors/legacy_executor.py: References "Layer 2 in workers/file_processing/structure_tool_task.py" — "Layer 2" doesn't appear anywhere in the codebaseworkers/file_processing/structure_tool_task.py: References "populated by Layer 1 export" — same issueworkers/file_processing/structure_tool_task.py(~line 670): Comment says "Use local variables so tool_metadata[_SK.OUTPUTS] is preserved for METADATA.json serialization downstream in _write_tool_result" — this is factually incorrect._write_tool_result()does not readtool_metadata[_SK.OUTPUTS]. The real reason is to feed only regular prompts intoanswer_paramswhile keeping the full list for the agentic dispatch loop.
7. complete_vision() docstring omits key behavioral differences from complete()
unstract/sdk1/src/unstract/sdk1/llm.py — around line 488
The docstring says "Same error handling, usage tracking, and metrics as complete()" but doesn't mention:
- Does NOT support
extract_jsonorpost_process_fnpost-processing - Does NOT prepend the adapter's system prompt (unlike
complete()which builds[{"role": "system", ...}, {"role": "user", ...}]internally)
Callers reading "same as complete()" might expect feature parity.
8. Significant test coverage gaps
No tests added for:
complete_vision()— 90-line new public method, zero coverage- Structure tool task partitioning/dispatch logic — core routing with zero tests
- IDE callback agentic table reshaping — 2-3 test cases needed in existing
TestIdePromptComplete - Legacy executor
agentic_tableguard — single test case needed
The IDE callback reshaping test is highest ROI: catches critical data-loss scenarios and the test infrastructure already exists in workers/tests/test_ide_callback.py.


What
AgenticTableSettingsCRUD backend (pluggable app) with per-prompt configuration for the extractor (LLM adapter, page range, parallel pages, highlight toggle)AgenticTableSettingsmodal for configuring the extractor andAgenticTableChecklistfor real-time prompt readiness validationWhy
agentic_tableHow
Backend
agentic_table_settings_v2pluggable app with model, serializer, views, URL routing, and validation serviceAgenticTableSettingsViewSet— full CRUD withupdate_or_createsemantics; returns saved instance (withid) so frontend can PATCHPromptValidationView— LLM-powered prompt analysis endpoint that checks whether a prompt contains target table, JSON structure, and instructions; usesget_or_createto avoid 404 chicken-and-egg issuesagentic_tableexecution payloads with adapter UUIDs from profileagentic_tablequeueCan this PR break any existing features? If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)
/prompt-studio/prompt/agentic-table/)agentic_tableenforce type — no existing enforce types are modifiedenforceType === "agentic_table"is selectedcreateview status code change (201 for new, 200 for update) is internal to this featureagentic_tabletype checkRelevant Docs
Related Issues or PRs
- UN-3403
Dependencies Versions / Env Variables
Notes on Testing
Backend Tests
Run the agentic table settings test suite:
Manual Testing
agentic_tableon a fresh prompt card -> type prompt -> verify no 404 -> configure LLM adapter -> verify checkboxes updateScreenshots
Attached in respective cloud PR.
...
Checklist
I have read and understood the Contribution Guidelines.