Skip to content

v1.3: fail-on-findings + in-band suppressions + first calibrated Terraform shape#65

Merged
Born14 merged 2 commits into
mainfrom
release/v1.3-suppressions
May 6, 2026
Merged

v1.3: fail-on-findings + in-band suppressions + first calibrated Terraform shape#65
Born14 merged 2 commits into
mainfrom
release/v1.3-suppressions

Conversation

@Born14
Copy link
Copy Markdown
Owner

@Born14 Born14 commented May 5, 2026

Summary

v1.3 adds three coupled features in response to engineering critique on v1.0-1.2 plus the first calibrated Terraform shape:

  1. fail-on-findings Action input (default false). Opt-in CI failure on non-suppressed findings. The receipt's calibrated severity stays at warning; the failure is the team's policy, not the methodology's claim.
  2. In-band suppression syntax (# verify:ignore <SHAPE-ID> reason:"<text>"). Trailing-only scope. Strict shape-ID matching (typos surface as warnings, not silent no-ops). Suppressions go in their own receipt block and are part of the packet digest.
  3. First calibrated Terraform shape (TF-SG-WORLD-OPEN-INGRESS-01): AWS security-group ingress opening sensitive ports to the world. Calibrated/warning under the v1.1 same-module Terraform resolver, promoted via strong-single-corpus on the devops-in-the-cloud corpus (32 TP / 0 FP / 16 AMB / 100% precision / 33.33% ambiguity).

Discipline reminder on the Terraform shape

Narrow but real: one shape, warning-only, single-corpus. The receipt's Not-Checked block names the rest of the Terraform surface as still uncovered. Not "Terraform solved", not blocking-tier, not two-corpus. Future Terraform shapes ship when each one earns its own ledger row.

Receipt schema (verify-receipt/v1) extended

Two new arrays, both part of the digest:

  • suppressions: SuppressedFinding[]
  • suppression_warnings: SuppressionWarning[]

Empty arrays when no suppressions are declared.

Surface extended to include terraform. CalibratedIaCShapeId extended to include TF-SG-WORLD-OPEN-INGRESS-01.

New canonical demo digest

sha256:0d7d9d941a672e37f2a31e0d8db8b20ce7ca416d57cf989c89deac7ca958b0e4

(Was sha256:bceed6d3... at v1.2; superseded the pre-Terraform v1.3 digest sha256:2040419... once TF-SG landed.)

Behaviour preserved on prior shapes

  • Same calibrated detectors for K8s/Dockerfile/GHA, same detection logic, same precision numbers, same ledger row provenance.
  • Receipt severity for the prior seven shapes unchanged.
  • Born14/verify@v1 install line unchanged.

What changed at the runtime

  • Eight calibrated checks shipping (was seven).
  • Bundle entry: src/action-v2/index-receipt.ts (the receipt entry, not the legacy migration entry).
  • New file detection: *.tf files now classify as the new terraform surface.
  • TF-SG runs extractTerraformSurface -> resolveSurface -> runTfSgWorldOpenIngress01 -> classifyFindings. Only TP findings appear as receipt findings; AMB findings are dropped (the substrate's resolution boundaries are named in the Not-Checked block).
  • Slim shipped calibration files (dist/action/calibration/{shapes.json,attempts.jsonl}) now contain 8 shapes / 11 ratified attempts.
  • Not-Checked Terraform line narrowed: "Terraform coverage is limited to TF-SG-WORLD-OPEN-INGRESS-01 ... Other Terraform shapes are not parsed or checked. Cross-module variables, expression evaluation, data sources, file()/templatefile(), and dynamic block expansion are out of scope."

Test plan

  • CI on this branch is green.
  • README.md on GitHub renders the eight-shape calibrated-checks table including the TF-SG row.
  • dist/action/index.cjs on GitHub contains TF-SG-WORLD-OPEN-INGRESS-01.
  • dist/action/index.cjs on GitHub does not contain the stale Terraform .tf files are not parsed.
  • Canonical demo digest in the release notes matches the rebuilt bundle/demo output.
  • Confirm a fixture with a real # verify:ignore comment routes the finding to suppressions and reduces non-suppressed count.
  • Confirm a fixture with a typo'd shape ID routes to suppression_warnings.
  • Smoke-test on Born14/verify-smoke once merged: install branch's bundle, observe receipt with eight checks and Terraform surface inspected.

Decisions locked in this release

  • Comment-walk scope: trailing-only.
  • Mismatched SHAPE-ID: strict match + warn-on-no-op.
  • fail-on-findings: true and suppressions: counts non-suppressed findings only.
  • Block ordering: position 5 (suppressions), between Not Checked and Reproduction.
  • TF-SG: ship TP findings only; drop AMB; preserve resolved_from provenance in evidence when same-module resolution backed the literal.
  • TF-SG severity: warning (first-promotion lock applies).

Not in this release (banked for later)

  • Full per-detector line-number plumbing.
  • Cryptographically-signed suppression comments.
  • Shape-specific suppression for CONTAINER-ROOT-01.
  • Second-corpus Terraform calibration (would unlock standard two-corpus path; deferred to a future release).
  • Cross-module Terraform variable resolution.
  • Other Terraform shapes (RDS hygiene, S3 versioning, etc.).

Tag move

The v1 tag is not moved by this PR. After merge, tag move requires explicit operator confirmation.

🤖 Generated with Claude Code

Two coupled changes that respond to real engineering critique on v1.0-1.2:
warnings get ignored in 2026 unless they have teeth, and intentional
exceptions need a first-class mechanism that doesn't break the audit chain.

Policy-driven blocking:
- New Action input fail-on-findings (default false). When true, the
  workflow exits 1 if there are non-suppressed findings.
- The receipt's calibrated severity is unchanged (warning). The
  receipt represents the methodology; the CI failure represents the
  user's policy. Severity belongs to the discipline; enforcement
  belongs to the team.
- Suppressed findings do not trigger the failure. They are still
  recorded in the receipt and covered by the digest.

In-band suppressions:
- New comment syntax: # verify:ignore <SHAPE-ID> reason:"<text>"
- Trailing-only scope: same line as trigger, or line immediately above.
  File-scoped and resource-scoped variants were rejected as abuse vectors.
- Strict shape-ID matching. Comments with typo'd or unknown shape IDs
  are recorded as suppression warnings, not silently treated as no-ops.
- Non-empty reason required. Empty-reason comments do not suppress.
- Suppressed findings move out of the primary findings block into a
  new MANIFEST INTENT / SUPPRESSIONS block (position 5 in the receipt).
- The packet digest covers the suppressions and warnings arrays. A
  suppression cannot be silently removed without changing the digest.

Receipt schema (verify-receipt/v1) extended with two arrays:
  - suppressions: SuppressedFinding[]
  - suppression_warnings: SuppressionWarning[]

Both are part of the digest. Empty arrays in the no-suppression case.

Bundle rebuilt with --legal-comments=none --minify-whitespace. Size:
234.6 KB (was 227 KB at v1.2). Behaviour: same 7 detectors, same
detection logic. Canonical demo digest with the v1.3 schema:
sha256:2040419351bc062a508ecac06f24d5c458db3e365d668168ae1967a6984f284d
(replaces the v1.2 digest sha256:bceed6d3...).

METHODOLOGY.md adds two sections: 'Methodology vs. policy' explaining
the severity-vs-enforcement separation, and 'Suppressions and intent'
documenting the comment syntax and the digest-covered audit chain.

The semantics:intent entry in the Not Checked block now reads: 'Verify
does not infer author intent. It only records intent that operators
declare explicitly via in-band suppression comments.'
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 5, 2026

VERIFY CHANGE RECEIPT
Kubernetes / Dockerfile / GitHub Actions
----------------------------------------------------------------
scope:   K8s 0 / Dockerfile 0 / GHA 0
checks:  8 calibrated
result:  CLEAR
digest:  sha256:1405d520aee006ea09afbf48d9b790114fa27edfaacce3f6a58130d9747770eb

Checks with no findings

  • [CLEAR] CONTAINER-ROOT-01
  • [CLEAR] K8S-MISSING-LIMITS-01
  • [CLEAR] K8S-MISSING-PROBES-01
  • [CLEAR] K8S-MISSING-SECURITY-CONTEXT-01
  • [CLEAR] K8S-IMAGE-TAG-LATEST-01
  • [CLEAR] GHA-SHA-PIN-01
  • [CLEAR] DOCKERFILE-BASE-IMAGE-DIGEST-UNPINNED-01
  • [CLEAR] TF-SG-WORLD-OPEN-INGRESS-01

Not checked

Terraform, CloudFormation, Helm-templated YAML, Kustomize overlays, runtime cloud state, business logic, intent, recall, uncalibrated shapes.

Receipt artifacts: verify-receipt.md, verify-receipt.json. Digest: sha256:1405d520aee006ea09afbf48d9b790114fa27edfaacce3f6a58130d9747770eb.

Eighth calibrated check shipping in the v1.3 receipt path:
TF-SG-WORLD-OPEN-INGRESS-01 (AWS security-group ingress opening
sensitive ports to the world). Calibrated/warning under the v1.1
Terraform substrate (same-module var/local resolver), promoted via
strong-single-corpus on the devops-in-the-cloud corpus.

Discipline reminder: this is one calibrated Terraform shape, narrow
but real, warning-only, single-corpus. The receipt's Not-Checked
block names the rest of the Terraform surface as still uncovered.
Not "Terraform solved", not blocking-tier, not two-corpus.

Bundle:
- dist/action/index.cjs rebuilt from the engine repo's
  src/action-v2/index-receipt.ts entry. Bundle bytes-identical to
  the engine-side bundle. Contains TF-SG-WORLD-OPEN-INGRESS-01 and
  the narrowed Not-Checked Terraform line; the stale "Terraform .tf
  files are not parsed" string is removed.
- dist/action/calibration/shapes.json + attempts.jsonl regenerated
  from the engine ledger; eight shipped shapes / eleven ratified
  attempts (was seven / ten).

Surface metadata:
- README.md table adds the TF-SG row; description and "What is not
  checked" block updated to reflect the narrow Terraform coverage.
- action.yml description updated to mention the Terraform shape.

Sample documentation:
- docs/VERIFY-RECEIPT-SAMPLE.md preserved verbatim as the v1 frozen
  sample (its sha256:bceed6d3... digest is an audit anchor); header
  relabeled "(historical, v1 schema; frozen)".
- docs/VERIFY-RECEIPT-SAMPLE-v1.3.md (new) is the current-state
  sample for the v1.3 schema with eight calibrated checks and the
  TF-SG row. Canonical demo digest:
  sha256:0d7d9d941a672e37f2a31e0d8db8b20ce7ca416d57cf989c89deac7ca958b0e4.
- docs/GITHUB-ACTION-MVP.md surface table, calibrated table, and
  Not-Checked block updated; the embedded v1 sample PR comment
  block is wrapped in a historical/frozen header.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Born14 Born14 changed the title v1.3: fail-on-findings policy lever + in-band suppressions v1.3: fail-on-findings + in-band suppressions + first calibrated Terraform shape May 6, 2026
@Born14 Born14 merged commit 2b1b979 into main May 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant