You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PR #33 fixed the open CodeQL code-scanning alerts (js/insecure-randomness, actions/code-injection) and, along the way, several pre-existing CI infra
bugs (Hypatia ${{ env.HOME }} path bug, Zig setup 404, Zig version, the erl_nif.h FFI include path). It is merged to main.
During that work three items were explicitly deferred as separate,
out-of-scope, by-design concerns rather than bugs. They are not blocking
anything today, but each is real technical debt or a latent gap. This issue
tracks doing them properly and in full. They are independent and can be
done in any order / separate PRs.
1. Hypatia findings never reach the code-scanning page (SARIF integration)
Current state
.github/workflows/hypatia-scan.yml:
permissions: contents: read only.
Produces hypatia-findings.json, uploads it as a 90-day artifact, and
(on PRs) posts a summary comment.
It does not convert findings to SARIF and does not upload them via
the code-scanning API. Only .github/workflows/codeql.yml
(security-events: write + github/codeql-action/analyze) feeds the
GitHub Security → Code scanning page.
Why it matters
This is the reason Hypatia "didn't catch" the alerts #33 fixed: even when
the scanner runs correctly, its findings are invisible on the code-scanning
surface, ungated, and not deduplicated/triaged like CodeQL alerts. It is a
scanner whose output nobody is required to look at.
Scope of work
Add a step that converts hypatia-findings.json → SARIF 2.1.0
(rule id, level mapped from severity, physicalLocation
file/region, message, partialFingerprints for stable dedupe).
Grant the job security-events: write (least privilege — only that
job/step, not the whole workflow if avoidable).
Upload via github/codeql-action/upload-sarif with a distinct category (e.g. hypatia) so it coexists with CodeQL results and
doesn't overwrite them.
Keep the existing artifact upload (useful for offline triage) but the
SARIF upload becomes the source of truth for the UI.
Decide & document the gating policy: advisory-only (informational
SARIF, no PR check failure) vs. fail-on-critical. Default
recommendation: advisory first, tighten later — but make it an
explicit, commented decision in the workflow, not an accident.
Handle the "scanner unavailable / empty findings" path: uploading an
empty SARIF run is fine and intentionally clears stale alerts; verify
it does not error.
Verify behaviour on PRs from forks (token will be read-only; upload-sarif from pull_request of a fork cannot write
security-events — document the limitation or move to workflow_run/default-branch upload).
Acceptance criteria
A deliberately-planted Hypatia finding appears as a code-scanning alert
with correct file/line/severity and a working "dismiss" flow.
An empty/again-clean scan resolves the alert rather than leaving it stale.
No regression to CodeQL alerts (separate category, both visible).
Gating policy is explicitly documented in the workflow.
2. Re-arm the Elixir test gate once the suite is green
Current state
.github/workflows/elixir-ci.yml, step "Run server tests" carries:
# GATE DEACTIVATED 2026-05-15 by maintainer request: the test# result no longer fails CI. ... This is a deliberate, temporary# suppression of signal, not a fix. To re-arm the gate, delete the# single line below.continue-on-error: true
The suite still runs and failures are visible, but the pass/fail gate is
severed. This is intentional and temporary, predicated on the suite being
red for a pre-existing reason unrelated to #33.
Blocking prerequisite
The actual mix test --no-start failure must first be root-caused and
fixed. Reproduction is currently blocked in two ways and needs ONE of:
widen the cloud-sandbox network allowlist to include repo.hex.pm + builds.hex.pm (Hex CDN; currently 403), or
enable a GitHub MCP "actions" logs tool / provide actions:read token, or paste the failing ** ( / N) test … failed block.
Static lead: server/test/test_helper.exs does Application.ensure_all_started(:burble) while CI runs mix test --no-start; if the :burble app fails to boot (NIF/priv artifact, port
bind, config) every test fails at startup — investigate the app-start
path first, not individual assertions.
Scope of work
Reproduce and root-cause the Test (OTP 27 / Elixir 1.17) failure.
Fix the underlying cause (separate PR from the gate change).
Once green on main, delete the single continue-on-error: true
line and confirm the gate fails on a deliberately-broken test.
Consider also gating the dialyzer job appropriately
(it needs: test).
Acceptance criteria
just test-server passes on main for OTP 27 / Elixir 1.17.
continue-on-error removed; a planted failing test turns CI red.
The "GATE DEACTIVATED" comment is removed with it.
3. Remove / wire up the dead "robot-repo-automaton (Phase 3)" autofix reference
Auto-fixable issues will be addressed by robot-repo-automaton (Phase 3)
There is no robot-repo-automaton / "Phase 3" automation wired into this
repo. The only auto-fixer present is rhodibot.yml, which does RSR
file-hygiene only — not security findings. The text promises an
auto-remediation path that does not exist, which is misleading in the scan
summary and step output.
Scope of work
Decide: either (a) remove the unfulfilled promise from the report
text and any sibling references, or (b) actually wire up a
finding→fix path (e.g. via the SARIF/issue output from item 1) if
that capability is intended.
If removing: also audit hypatia-scan.yml for the still-commented # exit 1 in "Check for critical issues" and the "Phase 2 /
gitbot-fleet" submission step — confirm those are intended and
documented, or clean them up too, so the workflow text matches
reality.
Acceptance criteria
The scan report / step summary describes only behaviour that actually
exists.
No dangling references to unimplemented "Phase 2/3" automation, or those
phases are genuinely implemented and linked.
Notes
Items 1 and 3 are closely related (a real SARIF/issue pipeline could
satisfy both); 2 is independent and has its own blocking prerequisite.
None of these regress anything currently on main; this is hardening and
truth-in-tooling, not firefighting.
Context
PR #33 fixed the open CodeQL code-scanning alerts (
js/insecure-randomness,actions/code-injection) and, along the way, several pre-existing CI infrabugs (Hypatia
${{ env.HOME }}path bug, Zig setup 404, Zig version, theerl_nif.hFFI include path). It is merged tomain.During that work three items were explicitly deferred as separate,
out-of-scope, by-design concerns rather than bugs. They are not blocking
anything today, but each is real technical debt or a latent gap. This issue
tracks doing them properly and in full. They are independent and can be
done in any order / separate PRs.
1. Hypatia findings never reach the code-scanning page (SARIF integration)
Current state
.github/workflows/hypatia-scan.yml:permissions: contents: readonly.hypatia-findings.json, uploads it as a 90-day artifact, and(on PRs) posts a summary comment.
the code-scanning API. Only
.github/workflows/codeql.yml(
security-events: write+github/codeql-action/analyze) feeds theGitHub Security → Code scanning page.
Why it matters
This is the reason Hypatia "didn't catch" the alerts #33 fixed: even when
the scanner runs correctly, its findings are invisible on the code-scanning
surface, ungated, and not deduplicated/triaged like CodeQL alerts. It is a
scanner whose output nobody is required to look at.
Scope of work
hypatia-findings.json→ SARIF 2.1.0(rule id, level mapped from
severity,physicalLocationfile/region, message,
partialFingerprintsfor stable dedupe).security-events: write(least privilege — only thatjob/step, not the whole workflow if avoidable).
github/codeql-action/upload-sarifwith a distinctcategory(e.g.hypatia) so it coexists with CodeQL results anddoesn't overwrite them.
SARIF upload becomes the source of truth for the UI.
SARIF, no PR check failure) vs. fail-on-critical. Default
recommendation: advisory first, tighten later — but make it an
explicit, commented decision in the workflow, not an accident.
empty SARIF run is fine and intentionally clears stale alerts; verify
it does not error.
upload-sariffrompull_requestof a fork cannot writesecurity-events — document the limitation or move to
workflow_run/default-branch upload).Acceptance criteria
with correct file/line/severity and a working "dismiss" flow.
2. Re-arm the Elixir test gate once the suite is green
Current state
.github/workflows/elixir-ci.yml, step "Run server tests" carries:The suite still runs and failures are visible, but the pass/fail gate is
severed. This is intentional and temporary, predicated on the suite being
red for a pre-existing reason unrelated to #33.
Blocking prerequisite
The actual
mix test --no-startfailure must first be root-caused andfixed. Reproduction is currently blocked in two ways and needs ONE of:
repo.hex.pm+builds.hex.pm(Hex CDN; currently 403), oractions:readtoken,or paste the failing
** (/N) test … failedblock.Static lead:
server/test/test_helper.exsdoesApplication.ensure_all_started(:burble)while CI runsmix test --no-start; if the:burbleapp fails to boot (NIF/priv artifact, portbind, config) every test fails at startup — investigate the app-start
path first, not individual assertions.
Scope of work
Test (OTP 27 / Elixir 1.17)failure.main, delete the singlecontinue-on-error: trueline and confirm the gate fails on a deliberately-broken test.
dialyzerjob appropriately(it
needs: test).Acceptance criteria
just test-serverpasses onmainfor OTP 27 / Elixir 1.17.continue-on-errorremoved; a planted failing test turns CI red.3. Remove / wire up the dead "robot-repo-automaton (Phase 3)" autofix reference
Current state
.github/workflows/hypatia-scan.yml("Generate scan report") prints:There is no
robot-repo-automaton/ "Phase 3" automation wired into thisrepo. The only auto-fixer present is
rhodibot.yml, which does RSRfile-hygiene only — not security findings. The text promises an
auto-remediation path that does not exist, which is misleading in the scan
summary and step output.
Scope of work
text and any sibling references, or (b) actually wire up a
finding→fix path (e.g. via the SARIF/issue output from item 1) if
that capability is intended.
hypatia-scan.ymlfor the still-commented# exit 1in "Check for critical issues" and the "Phase 2 /gitbot-fleet" submission step — confirm those are intended and
documented, or clean them up too, so the workflow text matches
reality.
Acceptance criteria
exists.
phases are genuinely implemented and linked.
Notes
satisfy both); 2 is independent and has its own blocking prerequisite.
main; this is hardening andtruth-in-tooling, not firefighting.
"why Hypatia didn't catch these" analysis.