Skill Being Reviewed
Skill name: iac-security
Skill path: skills/cloud/iac-security/
False Positive Analysis
Benign code that triggers a false positive:
Plan review treats unknown/sensitive/zero values as risk until source verified.
Why this is a false positive:
The current review guidance can push an agent to flag this pattern without first proving the attacker-controlled path, tenant boundary, or effective runtime behavior. The review should require evidence before classification.
Coverage Gaps
Missed variant 1:
Sensitive value is hidden in plan, reviewer assumes no secret changed.
Why it should be caught:
This is a realistic failure mode for iac-security. It changes the effective security boundary without matching the simpler examples currently emphasized by the skill.
Missed variant 2:
Provider default changes secret field to empty string, disabling auth.
Why it should be caught:
This variant commonly appears in production systems and needs explicit reviewer prompts so agents do not stop at static configuration review.
Edge Cases
Plan redaction can hide dangerous drift.
Remediation Quality
Comparison to Other Tools
| Tool |
Catches this? |
Notes |
| Semgrep |
Partial |
Can catch static patterns, but not effective policy, ownership, or runtime propagation without custom rules. |
| CodeQL |
Partial |
Strong for code/data-flow cases, weaker for cloud/control-plane and process evidence unless modeled. |
| Other: manual review |
Yes |
Human review can verify effective behavior, exception ownership, and operational evidence. |
Overall Assessment
Strengths:
The skill gives useful practitioner framing and asks for concrete evidence instead of generic advice.
Needs improvement:
Need secret-sensitive plan review gates.
Priority recommendations:
- Add a checklist item for secret zero-values in plan diffs evidence.
- Add one benign exception example so reviewers avoid noisy findings.
- Add one regression or verification step that proves the effective boundary after remediation.
Bounty Info
Skill Being Reviewed
Skill name: iac-security
Skill path:
skills/cloud/iac-security/False Positive Analysis
Benign code that triggers a false positive:
Why this is a false positive:
The current review guidance can push an agent to flag this pattern without first proving the attacker-controlled path, tenant boundary, or effective runtime behavior. The review should require evidence before classification.
Coverage Gaps
Missed variant 1:
Why it should be caught:
This is a realistic failure mode for iac-security. It changes the effective security boundary without matching the simpler examples currently emphasized by the skill.
Missed variant 2:
Why it should be caught:
This variant commonly appears in production systems and needs explicit reviewer prompts so agents do not stop at static configuration review.
Edge Cases
Plan redaction can hide dangerous drift.
Remediation Quality
Comparison to Other Tools
Overall Assessment
Strengths:
The skill gives useful practitioner framing and asks for concrete evidence instead of generic advice.
Needs improvement:
Need secret-sensitive plan review gates.
Priority recommendations:
Bounty Info
samik4184@gmail.com