test: trim fast gate to ~11s; run moved NNP tests in a slow CI job#99
Merged
Conversation
Mark the 8 tests that load the real AIMNet2 model via the session aimnet_model fixture as @pytest.mark.slow (5 adapter tests, 3 factory tests). With no real NNP load left, the fast gate (`pytest -m "not slow"`) drops from ~27s to ~11s and is fully deterministic (621 passed, all durations < 1.1s). To keep that real-NNP coverage in CI, add a `slow` job to tests.yml that installs the [ase,ani,dev] extras, caches the AIMNet2/torch model downloads, and runs the moved tests (test_model_adapter, test_model_factory, test_thermo_helpers) under `-m slow` on every push/PR. The job is deliberately scoped to those NNP integration tests rather than the full `-m slow` suite: the heavy full-pipeline tests (test_auto3D / test_SPE / test_thermo) pass individually but are flaky under combined, randomized ordering, so running them in CI as-is would be red. Hardening their isolation is tracked separately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #98: trims the fast gate to ~11s and recovers the real-NNP coverage in a dedicated CI job.
Fast gate: ~27s → ~11s
Marks the 8 tests that load the real AIMNet2 model (via the session
aimnet_modelfixture)@pytest.mark.slow— 5 intest_model_adapter.py, 3 intest_model_factory.py. With no real NNP load left,pytest -m "not slow"is 621 passed in ~11s, fully deterministic (all durations < 1.1s). That's ~9.5× faster than the original ~104s baseline.New
slowCI jobTo keep the real-NNP coverage running on every push/PR, adds a
slowjob totests.ymlthat:.[ase,ani,dev],~/.cache/aimnet) and torch/torchani (~/.cache/torch) model downloads,test_model_adapter.py,test_model_factory.py,test_thermo_helpers.pyunder-m slow.Verified locally: those 3 files under
-m sloware 10 passed, ~22s, deterministic across runs.Why scoped, not the full
-m slowsuiteA local run of the entire
-m slowsuite was 18 failed / 25 passed. The failures are pre-existing and unrelated to this change:ModuleNotFoundError: torchani(they pass with the[ani]extra the CI job installs);test_auto3D/test_SPE/test_thermo) that pass individually but fail under combined, randomized ordering (e.g.test_auto3D_config2passes alone in 18s). That's a test-isolation problem in the multiprocessing pipeline, tracked as separate hardening work — running the whole suite in CI as-is would be flaky-red.So this job runs exactly the deterministic NNP integration tests that moved out of the fast gate; the heavy suite stays out until its isolation is fixed.