Fix dragon kappa bisect failing for p >> n#388
Open
marouenbg wants to merge 1 commit into
Open
Conversation
The score-equation root for the kappa MLE in estimate_kappa and estimate_kappa_dragon scales like p*(p-1)/(4*|term_Dlogli|), which can exceed the static upper bound 1000*n when p is much larger than n. That makes scipy.optimize.bisect fail with "f(a) and f(b) must have different signs" because both endpoints stay positive. Add _bisect_growing_upper, which expands the upper bound geometrically until f(b) flips sign (the score equation asymptotes to a finite negative value, so a root always exists), and route the five bisect call sites through it. Repurpose test_catch_kappa_error -> test_kappa_dragon_high_p_low_n to check the call now succeeds, and add test_bisect_growing_upper_user_repro exercising the helper with the exact parameters from the bug report. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
estimate_kappaandestimate_kappa_dragonpreviously calledscipy.optimize.bisectwith the static bracket[1.001, 1000*n]. The score-equation root scales likep*(p-1)/(4*|term_Dlogli|), which can exceed1000*nwhenp ≫ n, producing the runtime errorf(a) and f(b) must have different signs._bisect_growing_upper, a small helper that grows the upper bound geometrically untilf(b)flips sign. The score equation asymptotes to a finite negative value asx → ∞, so the helper is guaranteed to bracket a root.dragon.py(estimate_kappaplus κ₁₁/κ₂₂/κ₁₂ and the simultaneous fit inestimate_kappa_dragon) through the new helper.Reproducer (from the bug report)
Asymptotic root location:
p1*(p1-1)/(4*|term|) ≈ 1.46×10⁶, well above1000*n = 8.32×10⁵. With the fix, one expansion (b → 8.32×10⁶) brackets the root and bisection converges.Tests
test_catch_kappa_error(which previously asserted the failure) is repurposed totest_kappa_dragon_high_p_low_n, asserting that the same(n=10, p1=p2=1000)call now returns finite positive κ values.test_bisect_growing_upper_user_reproreproduces the exact failing parameters from the report, asserts the original call still raises, and verifies the helper recovers a root agreeing with the asymptotic prediction to <0.1%.Test plan
pytest tests/test_dragon.py --deselect tests/test_dragon.py::test_dragon— 6 passed (the deselected test pulls fixtures from S3)test_dragon🤖 Generated with Claude Code