Preflight execution evidence before review#685
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (3)
Walkthroughprimary reviewer 호출 전에 실행 증거를 기계적으로 검증하는 preflight를 추가하고, 실패 시 개요이 PR은 primary reviewer 호출 전에 실행 증거( 변경사항실행 증거 사전검사 메커니즘
Sequence Diagram(s)sequenceDiagram
participant Runner
participant PreflightGuard
participant buildExecutionEvidencePreflight
participant relayEvents
Runner->>PreflightGuard: prepareOnly done -> maybeBlockForExecutionEvidencePreflight(runDir, reviewedHead, strict)
PreflightGuard->>buildExecutionEvidencePreflight: buildExecutionEvidencePreflight(runDir, reviewedHead, strict)
buildExecutionEvidencePreflight-->>PreflightGuard: { status, qualityExecutionStatus, reason, evidenceHeadSha, artifactPath, nextAction }
alt status == "pass"
PreflightGuard-->>Runner: allow primary reviewer invocation
else status == "blocked"
PreflightGuard->>relayEvents: appendRunEvent(REVIEW_PREFLIGHT_FAILED, { preflight_type, quality_execution_status, evidence_head_sha, ... })
PreflightGuard-->>Runner: block, emit JSON result, exitCode = 2
end
🎯 3 (Moderate) | ⏱️ ~25 분 가능한 관련 PR:
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Relay ReviewVerdict: LGTM |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/relay-review/scripts/review-runner/execution-evidence.js`:
- Around line 266-267: The code reads the execution evidence artifact into
artifactLoad then calls computeQualityExecutionStatus which re-reads the same
artifact, risking inconsistent snapshots; modify computeQualityExecutionStatus
to accept the already-read artifact (artifactLoad) or add a new parameter (e.g.,
evidenceArtifact) and use that instead of reading from disk, then update the
call site where artifactLoad is created (and any callers of
computeQualityExecutionStatus) so reason/qualityExecutionStatus and
evidenceHeadSha derive from the same in-memory snapshot.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 77070411-10c2-421a-8c09-130e7a73e6e9
📒 Files selected for processing (6)
skills/relay-review/references/runner-notes.mdskills/relay-review/scripts/review-runner.jsskills/relay-review/scripts/review-runner/execution-evidence.jsskills/relay-review/scripts/review-runner/preflight.jstests/relay-review/scripts/review-runner-execution-evidence.test.jstests/relay-review/scripts/review-runner.test.js
| const artifactLoad = readExecutionEvidenceArtifact(runDir); | ||
| const executionStatus = computeQualityExecutionStatus({ runDir, reviewedHead, strict }); |
There was a problem hiding this comment.
프리플라이트가 증거 파일을 두 번 읽어 결과 스냅샷이 엇갈릴 수 있습니다.
Line 266에서 아티팩트를 읽은 뒤, Line 267에서 computeQualityExecutionStatus가 같은 파일을 다시 읽습니다. 두 호출 사이에 파일이 바뀌면 reason/qualityExecutionStatus와 evidenceHeadSha가 서로 다른 스냅샷 기준으로 섞일 수 있습니다.
🔧 제안 패치
-function computeQualityExecutionStatus({ runDir, reviewedHead, strict = false }) {
- const artifactLoad = readExecutionEvidenceArtifact(runDir);
+function computeQualityExecutionStatus({
+ runDir,
+ reviewedHead,
+ strict = false,
+ artifactLoad: preloadedArtifact = null,
+}) {
+ const artifactLoad = preloadedArtifact || readExecutionEvidenceArtifact(runDir);
if (artifactLoad.state === "missing") {
return {
status: "missing",
@@
function buildExecutionEvidencePreflight({ runDir, reviewedHead, strict = false }) {
const artifactLoad = readExecutionEvidenceArtifact(runDir);
- const executionStatus = computeQualityExecutionStatus({ runDir, reviewedHead, strict });
+ const executionStatus = computeQualityExecutionStatus({
+ runDir,
+ reviewedHead,
+ strict,
+ artifactLoad,
+ });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skills/relay-review/scripts/review-runner/execution-evidence.js` around lines
266 - 267, The code reads the execution evidence artifact into artifactLoad then
calls computeQualityExecutionStatus which re-reads the same artifact, risking
inconsistent snapshots; modify computeQualityExecutionStatus to accept the
already-read artifact (artifactLoad) or add a new parameter (e.g.,
evidenceArtifact) and use that instead of reading from disk, then update the
call site where artifactLoad is created (and any callers of
computeQualityExecutionStatus) so reason/qualityExecutionStatus and
evidenceHeadSha derive from the same in-memory snapshot.
Relay ReviewVerdict: LGTM |
1 participant
Fixes #681
Refs #678
Summary
Adds execution-evidence preflight before primary reviewer invocation for mechanical evidence failures.
Relay
issue-681-20260607235606942-5c342363req-20260607235336010issue-681-execution-evidence-preflightbb24997d8dd8431bb199a2bc6c89aafae0aefab3Completion Audit
execution-evidence.jsonblocks before primary reviewer invocation.head_shadiffers from reviewed HEAD.--review-filefallback/fail-closed behavior remains intact.runner-notes.mddocuments script-level mechanical proof vs semantic AI review.Verification
node --test tests/relay-review/scripts/review-runner-execution-evidence.test.js tests/relay-review/scripts/review-runner.test.js tests/relay-review/scripts/review-runner-verdict.test.js-> 165/165 passexecution-evidence.jsonrebound by orchestrator recovery to HEADbb24997d8dd8431bb199a2bc6c89aafae0aefab3withtest_exit_code=0andverification_runs[].Notes
The initial relay-dispatch publication failed because the dispatch prompt said to commit to
issue-681while the manifest branch wasissue-681-preflight. The implementation commit was preserved,issue-681-preflightwas fast-forwarded to the implementation HEAD, and this PR was created as recovery.Summary by CodeRabbit
새로운 기능
행동/출력 변경
테스트
잡일