Draft
Conversation
Updated config to make sample-app run with parent Next.js 16
Heartbeat and more resilient implementation for DSRs. Extended queue visibility timeout on e-mail message service.
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
Dependency ReviewThe following issues were found:
Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned Files
|
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| "fides.api.tasks.queue_migration.get_sqs_client", | ||
| return_value=sqs_client, | ||
| ) | ||
| expect_exception = False |
| "fides.api.tasks.queue_migration.get_sqs_client", | ||
| return_value=sqs_client, | ||
| ) | ||
| expect_exception = False |
| "fides.api.tasks.queue_migration.get_sqs_client", | ||
| return_value=sqs_client, | ||
| ) | ||
| expect_exception = False # per-queue errors are swallowed |
| "fides.api.tasks.queue_migration.get_sqs_client", | ||
| side_effect=RuntimeError("cannot build sqs client"), | ||
| ) | ||
| expect_exception = False # SQS build errors are swallowed |
| from typing import Any, Dict, List, Optional | ||
| from unittest.mock import MagicMock, patch | ||
|
|
||
| import pytest |
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (41.57%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #7995 +/- ##
==========================================
- Coverage 84.97% 82.53% -2.44%
==========================================
Files 631 639 +8
Lines 41239 41680 +441
Branches 4787 4844 +57
==========================================
- Hits 35041 34402 -639
- Misses 5113 6158 +1045
- Partials 1085 1120 +35 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket ENG-3477
Description Of Changes
Migrate Fides' Celery task queue broker from Redis to Amazon SQS, eliminating Redis as a hard dependency for task queuing. Redis continues to serve as the application cache and Celery result backend; only the Celery broker role moves to SQS. The migration is controlled by the feature flag FIDES__QUEUE__USE_SQS_QUEUE (default false) so it can be toggled at runtime without a redeploy.
A zero-loss startup migration drains pending tasks from Redis queues and re-enqueues them to SQS during cutover, using a distributed SET NX EX lock to prevent duplicate processing.
Additionally adds a Queue Monitor page to the admin UI (/monitor/queues) for real-time visibility into SQS queue depths.
Code Changes
Steps to Confirm
Pre-Merge Checklist
CHANGELOG.mdupdatedmaindowngrade()migration is correct and works