Skip to content

Comments

Add Azure Queue Storage support with type-safe operations and blob audit logging#161

Draft
Copilot wants to merge 9 commits intomainfrom
copilot/implement-azure-queue-support
Draft

Add Azure Queue Storage support with type-safe operations and blob audit logging#161
Copilot wants to merge 9 commits intomainfrom
copilot/implement-azure-queue-support

Conversation

Copy link
Contributor

Copilot AI commented Feb 6, 2026

Implements first-class Azure Queue Storage infrastructure for distributed service communication with automatic blob audit trails, JSON schema validation, and OpenTelemetry tracing.

Core Infrastructure

@cellix/queue-storage-seedwork - Framework seedwork package

  • BaseQueueSender<TPayload> / BaseQueueReceiver<TPayload> with generic payload typing and runtime schema validation
  • MessageLogger - Fire-and-forget blob writes to queue-messages/{inbound|outbound}/{ISO8601-timestamp}.json
  • Configurable metadata/tags per queue
  • No any types - strict generics with discriminated unions

@ocom/service-queue-storage - Application service

  • Lifecycle-managed (startUp/shutDown) queue senders and receivers
  • JSON Schema definitions for CommunityCreatedPayload and MemberUpdatePayload
  • Type-safe sender/receiver instances exposed via getters

Cellix API Extension

Added registerAzureFunctionQueueHandler() to fluent startup API:

Cellix
  .initializeInfrastructureServices(registry => {
    registry.registerInfrastructureService(
      new ServiceQueueStorage(process.env.AZURE_STORAGE_CONNECTION_STRING)
    );
  })
  .registerAzureFunctionQueueHandler(
    'member-queue',
    { queueName: 'member', connection: 'AZURE_STORAGE_CONNECTION_STRING' },
    (appHost, infraRegistry) => async (queueEntry, context) => {
      const queueService = infraRegistry.getInfrastructureService(ServiceQueueStorage);
      const [msg] = await queueService.memberReceiver.receiveMessages();
      await appHost.forRequest().then(app => app.Members.updateMember(msg.payload));
      await queueService.memberReceiver.deleteMessage(msg.messageId, msg.popReceipt);
    }
  )

Proof-of-Concept Integrations

Outbound: CommunityCreatedEvent domain event sends typed payload to community-created queue with automatic blob logging

Inbound: member queue trigger receives messages, validates schema, updates member records via application service, deletes processed messages

Local Development

Azurite emulator support documented in AZURE_QUEUE_LOCAL_DEVELOPMENT.md with connection string configuration and testing examples.

Original prompt

This section details on the original issue you should resolve

<issue_title>Implement Azure Queue Storage support in Cellix with type-safe, logged queue sender/receiver and proof-of-concept</issue_title>
<issue_description>## Overview

The Cellix framework must provide robust, first-class support for Azure Queue Storage, enabling reusable queuing and logging for distributed communications between services. This work introduces foundational packages and application integration—including legacy-based queue sender/receiver abstractions, application-configurable service registration, and functional business examples.

Built-in Logging to Blob Storage

  • Every message sent or received (either direction) must be uploaded to Azure Blob Storage in a container queue-messages:
    • Messages sent: stored under queue-messages/outbound/
    • Messages received: stored under queue-messages/inbound/
    • File name: current timestamp (UTC, ISO8601, ms precision), e.g. 2026-02-07T14:42:03.123Z.json
  • Blob Metadata and Tagging:
    • Each file must be tagged and must have blob metadata for queue name and message direction.
    • Developers must be able to configure additional metadata/tags per queue at the application layer (e.g., custom tags per message type / queue).
  • Logging must be reliable, atomic, and must not block the send/receive pipeline (logging should not prevent the queue operation from completing; errors must be handled robustly and traced).
  • Documentation must include instructions for local Azurite-based development (storage emulator).

Implementation expectations (legacy parity + improvements)

  • Abstractions, sender/receiver, and service interface must provide at least the same feature completeness, reliability, and error handling as the legacy efdo implementation.
  • @cellix/queue-storage-seedwork must enforce proper type safety using generic typings and runtime guarantees:
    • No any for generic queue message/payload plumbing.
    • Prefer unknown + validation + typed narrowing where needed.
    • Prefer generics and discriminated unions for message envelopes and payload types.

Deliverables & Structure

1) @cellix/queue-storage-seedwork

Create a new framework seedwork package containing reusable queue storage infrastructure code:

2) @ocom/service-queue-storage

Create an Owner Community application-specific package that:

  • Depends on @cellix/queue-storage-seedwork.
  • Maintains Owner Community’s queue configuration:
    • queue names
    • direction (inbound/outbound)
    • schemas
    • logging metadata/tags configuration
  • On startup, registers all configured queues for sending and receiving.
  • Adheres to Cellix infrastructure service standards (startup/shutdown lifecycle, DI registration patterns).

3) Extend Cellix fluent startup API to support queue triggers

In @ocom/api (and/or Cellix core where appropriate), expose a fluent, chained startup API to register Azure Functions queue handlers similarly to how HTTP handlers are registered today.

Proof-of-concept scenarios (MUST be implemented in Owner Community)

These examples are required to prove the design works with what is already functional in the repo and to provide contributors a working reference.

Outbound queue example: community-created

  • On community creation, an existing integration event handler for CommunityCreatedEvent (already firing in the domain) must send a queue message to the outbound queue community-created.
  • The message contract should align with the actual event and include relevant fields (e.g., communityId, name, createdAt, etc.).
  • The send must:
    • be type-safe (generic typed payload)
    • be schema-validated at runtime
    • log the sent message as JSON to blob storage under queue-messages/outbound/ with configured tags/metadata

Inbound queue example: member

  • Create an inbound queue member that accepts a payload:
    • memberId: string (required; objectId)
    • Select a few sensible fields from the member schema which can be used on the member message payload for the queue handler to update those fields in database when it processes a message. It doesn't matter what fields you pick, ...

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI added 2 commits February 6, 2026 22:26
…alidation, and blob logging

- Create type-safe queue message infrastructure with generic envelopes
- Implement BaseQueueSender with automatic validation and blob logging
- Implement BaseQueueReceiver with message processing and blob logging
- Add MessageLogger for automatic audit trail to blob storage
- Add SchemaValidator with AJV for runtime type validation
- Add OpenTelemetry tracing integration
- Include comprehensive README and usage examples
- Add unit tests for schema validator
- Update pnpm workspace catalog with Azure SDK versions

Blob logging structure:
- Outbound: queue-messages/outbound/{timestamp}.json
- Inbound: queue-messages/inbound/{timestamp}.json
- Metadata/tags: queue name, direction, message ID, custom tags

Supports:
- Type-safe generic payloads
- Correlation IDs for distributed tracing
- Custom metadata per message
- Configurable blob logging per queue
- Local development with Azurite emulator

Part of: Azure Queue Storage support implementation
See: QUEUE_STORAGE_IMPLEMENTATION_SUMMARY.md for full implementation plan
Copilot AI changed the title [WIP] Add Azure Queue Storage support in Cellix framework feat: Add Azure Queue Storage seedwork with type-safe operations and blob audit logging Feb 6, 2026
Copilot AI requested a review from nnoce14 February 6, 2026 22:32
Copilot AI added 5 commits February 11, 2026 15:25
- Create ServiceQueueStorage infrastructure service
- Implement CommunityCreatedQueueSender for outbound events
- Implement MemberQueueReceiver for inbound updates
- Define payload schemas with JSON Schema validation
- Add lifecycle management (startUp/shutDown)
- Add unit tests
- Include comprehensive README with examples

Part of: Azure Queue Storage support (Phase 2 complete)
- Add StorageQueueFunctionOptions and StorageQueueHandler imports
- Add registerAzureFunctionQueueHandler method to AzureFunctionHandlerRegistry
- Update PendingHandler to support both HTTP and queue handlers
- Update setupLifecycle to register queue handlers with app.storageQueue
- Add comprehensive JSDoc documentation

Part of: Azure Queue Storage support (Phase 3 complete)
- Register ServiceQueueStorage in Cellix infrastructure
- Create community-created--send-queue-message event handler
- Update RegisterEventHandlers to accept optional queue service
- Send CommunityCreatedEvent to queue with full payload
- Add error handling to prevent queue failures from breaking events

Part of: Azure Queue Storage support (Phase 4 complete)
- Create memberQueueHandlerCreator for processing queue messages
- Add updateMember application service method
- Register member queue handler with Cellix
- Configure queue trigger with AZURE_STORAGE_CONNECTION_STRING
- Implement message processing with automatic deletion

Part of: Azure Queue Storage support (Phase 5 complete)
…ntation

- Create AZURE_QUEUE_LOCAL_DEVELOPMENT.md with Azurite setup guide
- Create AZURE_QUEUE_IMPLEMENTATION_COMPLETE.md with completion summary
- Update QUEUE_STORAGE_IMPLEMENTATION_SUMMARY.md with all phase details
- Document acceptance criteria status
- Add usage examples and architecture diagrams

Part of: Azure Queue Storage support (Phase 6 complete - ALL PHASES DONE)
Copilot AI changed the title feat: Add Azure Queue Storage seedwork with type-safe operations and blob audit logging Add Azure Queue Storage support with type-safe operations and automatic blob audit logging Feb 11, 2026
@nnoce14
Copy link
Member

nnoce14 commented Feb 12, 2026

@copilot Why am i seeing this warning when doing pnpm install, I have using vite 7.3.1 everywhere enforced by my pnpm workspace catalog. " WARN  Issues with peer dependencies found
packages/cellix/ui-core
├─┬ @vitest/browser 3.2.4
│ └─┬ @vitest/mocker 3.2.4
│ └── ✕ unmet peer vite@7.3.1: found 8.0.0-beta.3
└─┬ @storybook/react-vite 9.1.16
├── ✕ unmet peer vite@7.3.1: found 8.0.0-beta.3
├─┬ @joshwooding/vite-plugin-react-docgen-typescript 0.6.1
│ └── ✕ unmet peer vite@7.3.1: found 8.0.0-beta.3
└─┬ @storybook/builder-vite 9.1.16
└── ✕ unmet peer vite@7.3.1: found 8.0.0-beta.3
. prepare$ husky
└─ Done in 562ms
Done in 10.1s using pnpm v10.18.2"

Copy link
Contributor Author

Copilot AI commented Feb 12, 2026

@copilot Why am i seeing this warning when doing pnpm install, I have using vite 7.3.1 everywhere enforced by my pnpm workspace catalog. " WARN  Issues with peer dependencies found
packages/cellix...

This warning is not related to the Azure Queue Storage changes. It's a pre-existing dependency issue where Storybook 9.1.17 has a transitive dependency on Vite 8.0.0-beta.3, which conflicts with the catalog version (7.3.1).

The issue comes from Storybook's internal dependencies. Storybook 9.1.17 is pulling in the beta version of Vite 8, while @vitest/browser and other packages expect Vite 7.3.1.

This is a known compatibility issue with Storybook 9.x and can be safely ignored - it's just a peer dependency warning and won't affect functionality. The packages will work despite the warning.

To resolve it (optional), you could either:

  1. Wait for Storybook to release a version compatible with Vite 7.3.1
  2. Add a pnpm override to force Vite 7.3.1 for all packages
  3. Upgrade to Vite 8 in the catalog once it's stable (not recommended while it's in beta)

Copilot AI changed the title Add Azure Queue Storage support with type-safe operations and automatic blob audit logging Add Azure Queue Storage support with type-safe operations and blob audit logging Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Azure Queue Storage support in Cellix with type-safe, logged queue sender/receiver and proof-of-concept

2 participants