perf(worker): Batch testrun fetching for flake processing#829
perf(worker): Batch testrun fetching for flake processing#829sentry[bot] wants to merge 1 commit intomainfrom
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c2b07f2. Configure here.
| @sentry_sdk.trace | ||
| def process_single_upload( | ||
| upload: ReportSession, curr_flakes: dict[bytes, Flake], repo_id: int | ||
| upload: ReportSession, |
There was a problem hiding this comment.
Unused upload parameter after refactoring
Low Severity
The upload parameter of process_single_upload is no longer referenced anywhere in the function body. Previously it was used to call get_testruns(upload), but now that testruns are passed in directly, upload became dead code. It's still passed at the call site but serves no purpose inside the function.
Reviewed by Cursor Bugbot for commit c2b07f2. Configure here.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #829 +/- ##
=======================================
Coverage 92.25% 92.25%
=======================================
Files 1306 1306
Lines 48012 48016 +4
Branches 1636 1636
=======================================
+ Hits 44294 44298 +4
Misses 3407 3407
Partials 311 311
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Fixes WORKER-Y7Y. The issue was that: The
process_flakes_for_commitfunction executes a separate database query for testruns for each upload, causing an N+1 query pattern.get_testrunsto accept a list of upload IDs, enabling batched fetching of testruns.process_flakes_for_committo fetch all relevant testruns for a commit's uploads in a single database query.defaultdictfor efficient distribution to individual upload processing.process_single_uploadto eliminate redundant database calls within the loop.This fix was generated by Seer in Sentry, triggered automatically. 👁️ Run ID: 13387071
Not quite right? Click here to continue debugging with Seer.
Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
Note
Low Risk
Primarily a performance refactor that preserves existing flake/outcome logic, with low functional risk aside from potential edge cases in testrun grouping or increased memory use when materializing all results.
Overview
Optimizes flake processing to avoid an N+1 query pattern by fetching all recent
Testruns for a commit’s uploads in a single query, grouping them byupload_id, and passing the pre-fetched list intoprocess_single_upload.This changes
get_testrunsto accept multiple upload IDs (returning a materialized list) and updatesprocess_flakes_for_commit/process_single_uploadcall flow accordingly.Reviewed by Cursor Bugbot for commit c2b07f2. Bugbot is set up for automated code reviews on this repo. Configure here.