Baseline Workflow

Baseline Workflow

Track findings over time. The pattern: save a baseline today, fail CI only on new findings, fix at your own pace.

---

Why baselines

Real repos rarely start green. A 600-tool spec produces hundreds of findings on day one. Most are real, none are urgent enough to fix all at once. Baselines let you draw a line in the sand:

• Today's findings are accepted (matched).
• Tomorrow's new findings fail CI.
• Resolved findings (gone from the active list) are counted but not enforced.

Suppressions handle "we explicitly accept this forever, with reason." Baselines handle "we'll fix this eventually but not today."

Save a baseline

agents-shipgate baseline save -c shipgate.yaml \
  --out .agents-shipgate/baseline.json

This:

1. Runs a normal advisory scan.
2. Filters to active, unsuppressed findings only.
3. Writes their fingerprints + minimal metadata to .agents-shipgate/baseline.json.

Commit the baseline:

git add .agents-shipgate/baseline.json
git commit -m "Baseline shipgate findings (147 critical, 663 high)"

The baseline file is human-readable JSON:

{
  "schema_version": "0.2",
  "project": { "name": "digitalocean-ops" },
  "agent": { "name": "do-ops-agent" },
  "created_at": "2026-04-26T14:18:09Z",
  "source_report_run_id": "agents_shipgate_…",
  "findings": [
    {
      "fingerprint": "fp_4cdcf5b467f27a65",
      "check_id": "SHIP-POLICY-APPROVAL-MISSING",
      "tool_name": "delete_v2_droplets_droplet_id",
      "severity": "critical",
      "title": "delete_v2_droplets_droplet_id lacks a declared approval policy"
    }
  ]
}

Apply a baseline

agents-shipgate scan -c shipgate.yaml \
  --baseline .agents-shipgate/baseline.json \
  --ci-mode strict --fail-on critical,high

Each finding in the report now carries baseline_status:

• matched — fingerprint exists in the baseline; not counted as new
• new — finding emerged since the baseline was saved
• resolved — fingerprint was in the baseline but no longer fires (counted in baseline.resolved_count)

In strict mode with a baseline applied, CI fails only on new findings that hit fail_on.

The summary section prints:

Baseline: matched=812, new=3, resolved=14

What's in a fingerprint

fingerprint = "fp_" + sha256(check_id | tool_name | canonical_evidence)[:16]

Stable across runs. Specifically excluded:

• Severity (so severity_overrides don't invalidate baselines)
• Source paths and line numbers (so refactors don't break matching)
• Timestamps and run IDs
• default_severity audit-evidence key (the override-tracking field)

The tradeoff: changing the structure of a check's evidence dict in a future shipgate version will rotate fingerprints. We don't break this lightly. The report_schema_version bumps when we do.

When to refresh

Situation	Action
Found a new false positive	Add a checks.ignore entry with a reason; do not re-baseline
Fixed several findings	Re-baseline so resolved ones stop showing up; commit the new file
Upgraded to a new agents-shipgate version with new checks	Strict-mode CI will fail on new check IDs as expected — fix or suppress, then re-baseline
Added new tools to the surface	New tools' findings are new and will fail strict CI — fix or accept, then re-baseline

Re-baselining is just agents-shipgate baseline save again. Diff the new file vs. the old in code review.

Suppressions vs baselines

Pick the right tool:

	Suppression (checks.ignore)	Baseline
Intended duration	Permanent	Until fixed
Requires reason	Yes	No (file metadata is the audit trail)
Affects severity counts	Removed from active counts	Stays in counts; tagged matched
Survives manifest changes	Yes (matches by check_id + tool_name)	Yes (matches by fingerprint)
Best for	"This is fine because…"	"Big cleanup project; ratchet over time"

If you find yourself baselining the same finding for months with no plan to fix it, convert it to a suppression with a real reason.

CI integration

Add the baseline to your action invocation:

- uses: ThreeMoonsLab/agents-shipgate@v0.2.0
  with:
    ci_mode: strict
    fail_on: critical,high
    baseline: .agents-shipgate/baseline.json

Or via the CLI in any other CI:

agents-shipgate scan -c shipgate.yaml \
  --ci-mode strict --fail-on critical,high \
  --baseline .agents-shipgate/baseline.json

The action surfaces baseline_new_count as an output you can wire into Slack/email notifiers.

Limitations (v0.2)

• --baseline-mode accepts only new-findings today (the flag is hidden in CLI help). More modes (e.g. severity-floor diff) are tracked for v0.3.
• Resolved findings appear only as a count in the summary, not enumerated. Diff the baseline JSON in code review to see which fingerprints disappeared.
• Multi-config workspace scans share a single baseline path. To use per-config baselines, run each scan separately.