Closed
Conversation
The v3 refactor removed the hardcoded gpt-5.2/high fallback from review-validator.ts but didn't add REVIEW_MODEL and REASONING_EFFORT to the Prepare validator step's env block in action.yml. This causes the validator pass to fall back to the Droid CLI's internal default model instead of using the user's configured review_model.
Read .factory/skills/review-guidelines.md from the workspace and inject its content into all review prompt templates (code review, candidates, validator, and security review). This allows repository maintainers to define repo-specific review guidelines without polluting AGENTS.md. Closes FAC-16667 Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
- Use .factory/skills/review-guidelines/SKILL.md path (follow skill conventions) - Extract formatGuidelinesSection() helper to centralize prompt formatting - Use consistent <custom_review_guidelines> tags across all templates Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
feat: support custom review guidelines via review-guidelines skill
Truncate review guidelines exceeding 80k characters (matching AGENTS.md handling) and append a note directing the model to read the full file. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
- Reserve space for truncation marker within the size limit (total stays <= MAX) - Add minimum meaningful space check (>200 chars) - Nudge model to use tools to read full file when truncated Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
feat: add size cap and truncation for review guidelines
…PR review body The review previously posted two comments: the initial tracking comment (updated with completion status) and a separate PR review summary via submit_review. This consolidates them so the review summary is written to the tracking comment using update_droid_comment, and submit_review is called without a body parameter. Applies to both single-pass review and two-pass validator flows. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
The single-pass review flow was initially created because we were unsure about the performance of the two-pass validator flow. The two-pass flow has since become the default and proven itself, and the single-pass flow is no longer used. This removes the review_use_validator toggle and all single-pass review code. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…y-comment Consolidate review summary into tracking comment
…s-review # Conflicts: # src/create-prompt/templates/review-prompt.ts # test/create-prompt/templates/review-prompt.test.ts
Remove single-pass review flow, always use two-pass validator
Instead of posting inline comments individually via create_inline_comment (which creates separate review events for each), batch all comments into the submit_review comments array. This produces a single cohesive review. - Updated submit_review tool schema to support line/side/start_line format - Updated review and validator prompts to batch comments - Removed create_inline_comment from review allowed tools Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Batch all review comments into a single submit_review call
…ation Remove direct injection of review-guidelines file content into orchestrator prompts. Instead, the file-group-reviewer subagent now invokes the review-guidelines skill via the Skill tool, ensuring the guidelines are consumed where they're actually needed (by the reviewer, not the orchestrator). Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…ines The --enabled-tools allowlist passed to droid exec did not include Skill, which meant file-group-reviewer subagents could not invoke the review-guidelines skill even though their droid config listed it. Add Skill to the candidateGenerationTools array in both generate-review-prompt and review command. Also strengthen file-group-reviewer instructions to treat custom guideline violations as mandatory (not stylistic) and make skill invocation the required first step. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
refactor: move review guidelines from prompt injection to skill invocation
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix(review): escape suggestion blocks in prompt Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix(review): enforce suggestion-only replacements Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> update
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…ions parameter Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Feat/review suggestion blocks
Load the review skill's shared methodology from factory-mono's builtin-skills/review/SKILL.md instead of keeping it inline in the CI prompt templates. The skill is loaded at runtime via local plugin cache or GitHub fallback. The shared methodology (bug patterns, reporting gate, confidence calibration, deduplication, analysis discipline) is extracted via BEGIN_SHARED_METHODOLOGY / END_SHARED_METHODOLOGY markers and injected into both candidate and validator prompts. Suggestion block rules remain controlled by the include_suggestions toggle at the CI template level. Depends on: Factory-AI/factory-mono#11498 Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
The Droid CLI bundles the review skill as a builtin, so the agent can invoke it via the Skill tool at runtime. Remove the load-skill.ts GitHub fetch and instead instruct the agent to invoke the 'review' skill directly. This eliminates the network dependency during the prepare step and uses the CLI as the single source of truth for the review methodology. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…skill Both candidate and validator prompts now only provide: - Session framing (senior engineer / validating candidates) - PR context and file paths - Skill invocation instruction (Pass 1 or Pass 2) - CI-specific output schema and posting constraints All review methodology, triage, parallel review, validation rules, deduplication, and confidence filtering come from the skill. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Allows using a custom CLI build for testing builtin skills from unreleased branches. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Not needed for this PR. Can be added separately if custom CLI testing is needed in the future. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
feat: load review methodology from factory-mono builtin skill
feat: add shallow/deep review depth presets (default: shallow with kimi-k2-0711)
chore: Security review plugins
fix(cli): Security review plugins prompt update
STRIDE enforcement for first step of security workflow
When both automatic_review and automatic_security_review were true, prepareTagExecution returned early without calling prepareReviewMode, so no prompt file was written. The Droid Exec step then failed with 'Prompt file does not exist'. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…eview-flags fix: create prompt file when both automatic review flags are set
…ew (#71) * feat: move security review to skill + run concurrently with code review - Create security-reviewer custom droid that invokes the security-review skill and returns findings as a JSON array (same format as file-group-reviewer) - Simplify security-review-prompt.ts to a candidates prompt that references the security-review skill instead of inlining all STRIDE methodology - When both automatic_review and automatic_security_review are enabled, spawn security-reviewer as a Task subagent alongside file-group-reviewers during pass 1, merging all candidates into one JSON for the validator - Standalone @droid security now uses the two-pass pipeline (candidates + validator) instead of posting inline comments directly - Update security/action.yml with validator steps - Enable Task/Skill/FetchUrl tools for security review candidate generation Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> * test: point droid-review workflow at branch for CI testing Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> * feat: use [security] tag instead of STRIDE letters, add security badge to tracking comment Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> * fix: show correct tracking comment when running both code review and security review Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --------- Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
The internal droid-review.yml workflow passes automatic_security_review: true, but @V3 does not support that input. The @V3 prepare step enters the combined review code path which returns early without generating a prompt file, causing the base-action to fail with 'Prompt file does not exist'. Pointing at @dev picks up the concurrent security review support. This only affects CI for this repo -- customers continue using @V3 in their own workflows. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
* eval: remove file-group-reviewer droid in favor of SKILL.md inlined methodology The review skill now uses worker subagents with the full shared methodology inlined into each prompt. The file-group-reviewer.md droid and its tests are no longer needed. * chore: update version refs to v5, clean up file-group-reviewer references - Update all @V3 references to @v5 in README and droid.yml workflow - Remove file-group-reviewer mentions from prompts, comments, and droid descriptions Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --------- Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.