Add MOT format file loading and sequence preparation for evaluation (TrackEval Part 4) #214
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add MOTFrameData dataclass for per-frame detection data - Add MOTSequenceData dataclass for prepared sequence data ready for metrics - Add load_mot_file() to parse MOT Challenge format files - Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation - Update __init__.py to export new types and functions - Add comprehensive unit tests (20 tests for io module)
SkalskiP
changed the title
- Add test/conftest.py with fixture to download and cache SoccerNet test data - Add test/eval/test_integration.py with 49 parametrized tests for all sequences - Add ci-integration-tests.yml workflow that runs only on eval code changes - Update ci-tests.yml to exclude integration tests from regular CI - Add integration marker to pyproject.toml pytest config All 49 sequences pass with exact numerical parity to TrackEval for: - Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag - Float metrics: MOTA, MOTP, MTR, PTR, MLR
…o trackeval-integration-part4
SkalskiP
force-pushed
the
trackeval-integration-part4
branch
from
cd12d96 to
aa3e0d2
Compare
- Rename CI job to 'TrackEval Parity Validation' for clarity - Derive sequence names dynamically from expected_results.json - Simplify test_io.py from 21 to 10 test cases while maintaining coverage - Restore useful comments in io.py around ID remapping and IoU computation - Clean up conftest.py and test_integration.py docstrings
SkalskiP
force-pushed
the
trackeval-integration-part4
branch
from
9fc42b3 to
e1536bc
Compare
* Update integration tests to use SportsMOT and DanceTrack data
Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack
(25 sequences) datasets. The tests now validate CLEAR metrics against
TrackEval results for 70 total sequences.
- Update URLs to new GCS-hosted test data zips
- Add multi-dataset fixture support in conftest.py
- Parametrize tests across both datasets
- Fix metric comparison (new format uses fractions, not percentages)
* Fix mypy conftest module name conflict
Use a pytest fixture instead of directly importing from conftest.py to avoid
mypy seeing the file under two module names ("conftest" and "test.conftest").
* Add type annotation for test_cases variable
tstanczyk95
approved these changes
Feb 3, 2026
Collaborator
tstanczyk95
left a comment
tstanczyk95
left a comment
There was a problem hiding this comment.
It was discussed in detailed and demonstrated. Evaluations on datasets with more shaky detections performed as requested and the results stayed consistent.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add MOT Challenge format file loading and sequence preparation functionality to enable end-to-end evaluation workflows. This PR introduces the data I/O layer that bridges file-based tracking data with the evaluation metrics.
Data I/O
MOTFrameData- Dataclass containing per-frame detection data (IDs, boxes, confidences, classes)MOTSequenceData- Dataclass containing prepared sequence data ready for metric evaluation (with 0-indexed IDs, IoU similarity matrices, and counts)load_mot_file()- Parse MOT Challenge format files (comma/space separated: frame, id, x, y, w, h, conf, class, ...)prepare_mot_sequence()- Compute IoU similarities and remap IDs to 0-indexed contiguous values as required by CLEAR/HOTA/Identity metricsIntegration Tests