Skip to content

2025 10 working branch#72

Open
simonvanlierde wants to merge 103 commits intomainfrom
2025-10_working_branch
Open

2025 10 working branch#72
simonvanlierde wants to merge 103 commits intomainfrom
2025-10_working_branch

Conversation

@simonvanlierde
Copy link
Contributor

@simonvanlierde simonvanlierde commented Nov 8, 2025

  • Backend:
    • Fixed backup scripts
    • Move to mjml email templates and fastapi-mail over custom smtp setup
    • Using Redis for disposable email checking cache, can use later for session mgmt
    • Fixed pydantic 2.12 compatibility issues
    • Add order_by in products endpoint
    • Linting and other small fixes
  • Frontend
    • Fix registration flow
    • Remove redundant hardcoded data (products.json, data.json)
    • Fix issue where product cannot be saved even on valid product
    • Add tooltip on save button showing validation issues

Copilot AI review requested due to automatic review settings November 8, 2025 10:32
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces significant infrastructure improvements and refactoring across the ReLab application stack:

Purpose: Add Redis caching support, improve email handling with MJML templates, enhance validation, update dependencies, and refactor database models for Pydantic 2.12+ compatibility.

Key Changes:

  • Infrastructure: Added Redis cache service with health checks and persistence
  • Email System: Migrated from plain text to MJML-compiled HTML templates with FastAPI-Mail integration
  • Dependencies: Updated Expo/Metro/React ecosystem, upgraded Pydantic constraints, added Redis/MJML libraries
  • Database: Fixed SQLModel relationship issues for Pydantic 2.12+ compatibility
  • Validation: Improved frontend user/product validation with better error messages

Reviewed Changes

Copilot reviewed 87 out of 92 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
renovate.json Formatting changes and added :preserveSemverRanges preset
frontend-web/package.json Updated expo-image from ~2.3.0 to ~2.4.0
frontend-web/package-lock.json Dependency updates for Expo/Metro ecosystem and peer dependencies
frontend-app/package.json Updated expo from 54.0.13 to 54.0.15
frontend-app/package-lock.json Updated Expo dependencies and added yaml package
frontend-app/src/services/api/validation/*.ts New validation utilities with structured error messages
frontend-app/src/components/product/ProductComponents.tsx Updated to use new validation functions
frontend-app/src/app/products/[id]/index.tsx Added tooltip for validation errors and useMemo
frontend-app/src/app/(auth)/new-account.tsx Complete refactor with real-time validation and improved UX
compose.yml Added Redis service with health checks and updated image digests
compose.prod.yml Added cache volume persistence and updated backup compression
compose.override.yml Exposed Redis port 6379 for development
backend/pyproject.toml Added Redis, FastAPI-Mail, MJML; upgraded Pydantic/SQLModel
backend/app/core/config.py Added Redis settings and converted passwords to SecretStr
backend/app/core/redis.py New Redis connection management with graceful degradation
backend/app/main.py Added lifespan manager for Redis and email checker initialization
backend/app/templates/emails/src/*.mjml New MJML email templates for all email types
backend/app/templates/emails/build/*.html Compiled HTML email templates
backend/tests/conftest.py Added email testing fixtures and mock utilities
backend/tests/tests/emails/*.py New comprehensive email tests
backend/scripts/seed/migrations_entrypoint.sh Improved environment variable handling with lowercase helper
backend/scripts/create_superuser.py Fixed to use SecretStr.get_secret_value()
backend/scripts/compile_email_templates.py New script to compile MJML templates
backend/scripts/backup/*.sh New backup scripts for PostgreSQL and user uploads with rsync/rclone support
backend/app/api/*/models.py Added explicit relationship kwargs for Pydantic 2.12+ compatibility
Files not reviewed (2)
  • frontend-app/package-lock.json: Language not supported
  • frontend-web/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 87 out of 92 changed files in this pull request and generated 2 comments.

Files not reviewed (2)
  • frontend-app/package-lock.json: Language not supported
  • frontend-web/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 87 out of 92 changed files in this pull request and generated 3 comments.

Files not reviewed (2)
  • frontend-app/package-lock.json: Language not supported
  • frontend-web/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@simonvanlierde
Copy link
Contributor Author

simonvanlierde commented Nov 17, 2025

@mrvisscher I just added a basic circularity_properties model (7e614f5), I think we can start with implementing this in the frontend.

namespace: Cache namespace to clear (e.g., "background-data", "docs")
"""
await FastAPICache.clear(namespace=namespace)
logger.info("Cleared cache namespace: %s", namespace)

Check failure

Code scanning / CodeQL

Log Injection High

This log entry depends on a
user-provided value
.

Copilot Autofix

AI 3 days ago

In general, to fix log injection issues, you should sanitize or normalize any user-controlled data before including it in log messages. For plain-text logs, a standard mitigation is to strip or replace newline and carriage-return characters (and optionally other non-printable characters) so that user-supplied values cannot break the log format or introduce extra lines.

The best targeted fix here is to ensure that namespace is sanitized inside clear_cache_namespace before it is logged. This keeps the external API and behavior of clear_cache_namespace unchanged for callers, while ensuring that, regardless of the type or validation performed earlier, the value written into the logs cannot contain dangerous line-break characters. Concretely, we can introduce a local variable such as safe_namespace that replaces \r\n, \r, and \n with empty strings, and log safe_namespace instead of the original namespace. Since we’re only touching the logging call and not the FastAPICache.clear invocation, there’s no impact on caching functionality.

All required changes are confined to backend/app/core/cache.py around the clear_cache_namespace function. No new imports are necessary; we can use Python string methods directly.

Suggested changeset 1
backend/app/core/cache.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/backend/app/core/cache.py b/backend/app/core/cache.py
--- a/backend/app/core/cache.py
+++ b/backend/app/core/cache.py
@@ -231,4 +231,5 @@
         namespace: Cache namespace to clear (e.g., "background-data", "docs")
     """
     await FastAPICache.clear(namespace=namespace)
-    logger.info("Cleared cache namespace: %s", namespace)
+    safe_namespace = namespace.replace("\r\n", "").replace("\r", "").replace("\n", "")
+    logger.info("Cleared cache namespace: %s", safe_namespace)
EOF
@@ -231,4 +231,5 @@
namespace: Cache namespace to clear (e.g., "background-data", "docs")
"""
await FastAPICache.clear(namespace=namespace)
logger.info("Cleared cache namespace: %s", namespace)
safe_namespace = namespace.replace("\r\n", "").replace("\r", "").replace("\n", "")
logger.info("Cleared cache namespace: %s", safe_namespace)
Copilot is powered by AI and may make mistakes. Always verify output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants