Skip to content

Fix dragon kappa bisect failing for p >> n#388

Open
marouenbg wants to merge 1 commit into
netZoo:develfrom
marouenbg:fix-dragon-kappa-bracket
Open

Fix dragon kappa bisect failing for p >> n#388
marouenbg wants to merge 1 commit into
netZoo:develfrom
marouenbg:fix-dragon-kappa-bracket

Conversation

@marouenbg
Copy link
Copy Markdown
Contributor

Summary

  • estimate_kappa and estimate_kappa_dragon previously called scipy.optimize.bisect with the static bracket [1.001, 1000*n]. The score-equation root scales like p*(p-1)/(4*|term_Dlogli|), which can exceed 1000*n when p ≫ n, producing the runtime error f(a) and f(b) must have different signs.
  • Add _bisect_growing_upper, a small helper that grows the upper bound geometrically until f(b) flips sign. The score equation asymptotes to a finite negative value as x → ∞, so the helper is guaranteed to bracket a root.
  • Route the five bisect call sites in dragon.py (estimate_kappa plus κ₁₁/κ₂₂/κ₁₂ and the simultaneous fit in estimate_kappa_dragon) through the new helper.

Reproducer (from the bug report)

p1 = 21337, n = 832, term_Dlogli11 = -77.92618920329457
Dlogli11(1.001)   = +2.27e11   ← positive
Dlogli11(1000*n)  = +58.87     ← still positive, bracket invalid

Asymptotic root location: p1*(p1-1)/(4*|term|) ≈ 1.46×10⁶, well above 1000*n = 8.32×10⁵. With the fix, one expansion (b → 8.32×10⁶) brackets the root and bisection converges.

Tests

  • test_catch_kappa_error (which previously asserted the failure) is repurposed to test_kappa_dragon_high_p_low_n, asserting that the same (n=10, p1=p2=1000) call now returns finite positive κ values.
  • New test_bisect_growing_upper_user_repro reproduces the exact failing parameters from the report, asserts the original call still raises, and verifies the helper recovers a root agreeing with the asymptotic prediction to <0.1%.

Test plan

  • pytest tests/test_dragon.py --deselect tests/test_dragon.py::test_dragon — 6 passed (the deselected test pulls fixtures from S3)
  • CI runs the full suite, including test_dragon

🤖 Generated with Claude Code

The score-equation root for the kappa MLE in estimate_kappa and
estimate_kappa_dragon scales like p*(p-1)/(4*|term_Dlogli|), which can
exceed the static upper bound 1000*n when p is much larger than n.
That makes scipy.optimize.bisect fail with "f(a) and f(b) must have
different signs" because both endpoints stay positive.

Add _bisect_growing_upper, which expands the upper bound geometrically
until f(b) flips sign (the score equation asymptotes to a finite negative
value, so a root always exists), and route the five bisect call sites
through it. Repurpose test_catch_kappa_error -> test_kappa_dragon_high_p_low_n
to check the call now succeeds, and add test_bisect_growing_upper_user_repro
exercising the helper with the exact parameters from the bug report.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant