Skip to content

Verify indirect deps reachable on incremental run#20735

Open
ilevkivskyi wants to merge 2 commits intopython:masterfrom
ilevkivskyi:fix-indirect-unreachable
Open

Verify indirect deps reachable on incremental run#20735
ilevkivskyi wants to merge 2 commits intopython:masterfrom
ilevkivskyi:fix-indirect-unreachable

Conversation

@ilevkivskyi
Copy link
Member

@ilevkivskyi ilevkivskyi commented Feb 4, 2026

Fixes #19477

This is another surprising correctness hole in the indirect dependencies logic. We implicitly assume everywhere that indirect dependencies are subset of transitive ones. Although it is true when the indirect dependencies are initially calculated, there is no guarantee this will stay true after an incremental update where some import structure has changed.

Currently, we only handle the situations where an indirect dependency is removed from the graph completely, but if it is still in the graph, but now brought in by a different dependent, this may cause out-of-order cache loading, and thus a bug or a crash. It is interesting that although in theory it is kind of a big deal, in practice it is very hard to create such scenario. So far I found only two complicated scenarios (see tests).

Implementation note: although this additional check is needed for rare edge cases, it is quite computationally expensive. I tried few options (including only computing import deltas), but the only viable/robust option seems to be an honest recursive check with some caching. I tested this on torch (it has ~1000 small SCCs plus single SCC with around 1000 modules), and the performance penalty is negligible there. So hopefully it should be fine even for large code bases.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Startup error: occasional "AssertionError: Cannot find component 'SSL' for 'OpenSSL.SSL.Context'"

1 participant