Custom collections on entity streams + session comments#4538
Open
balegas wants to merge 4 commits into
Open
Conversation
Contributor
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #4538 +/- ##
==========================================
- Coverage 56.45% 55.44% -1.02%
==========================================
Files 358 319 -39
Lines 39081 37103 -1978
Branches 10973 10700 -273
==========================================
- Hits 22064 20570 -1494
+ Misses 16946 16498 -448
+ Partials 71 35 -36
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
Electric Agents Mobile BuildLocal mobile checks ran for commit The EAS Android preview build was skipped because the |
balegas
force-pushed
the
custom-collections
branch
from
485264e to
744274a
Compare
KyleAMathews
reviewed
Jun 9, 2026
| ...(definition.inboxSchemas && { | ||
| inbox_schemas: mapSchemas(definition.inboxSchemas), | ||
| }), | ||
| ...(definition.customCollectionSchemas && { |
Contributor
There was a problem hiding this comment.
you can already define custom schemas so we don't need this
Entity types declare the custom collections they accept by registering a
JSON Schema for each one in `custom_collection_schemas`, mirroring the
existing `inbox_schemas` / `state_schemas` pattern. Writes to a custom
collection flow through one generic endpoint that always validates
against the declared schema before appending.
Server: `POST /_electric/entities/:type/:id/collections/:name` accepts
opaque JSON, looks up the schema declared for `<name>` on the entity's
type, and rejects writes (422) when no schema is declared or the value
doesn't validate. Reserved built-in collection names
(`BUILT_IN_COLLECTION_TYPES`) are rejected here too so the runtime
stays the sole writer of agent-managed collections.
- New `custom_collection_schemas` JSONB column on `entity_types` and
`entities` (migration 0015).
- Types: `ElectricAgentsEntityType`, `ElectricAgentsEntity`,
`PublicElectricAgentsEntity`, `RegisterEntityTypeRequest`, and the
observation membership row carry the new field.
- Registry CRUD plus `updateEntityTypeInPlace` / `createEntity` /
`rowToEntityType` / `rowToEntity` plumb it through.
- Entity manager: `registerEntityType` validates the map, `spawn`
snapshots type → entity, `getEffectiveSchemas` returns the merged
map, `appendCollectionRow` validates against it, `amendSchemas`
supports additive amendments via a shared `mergeAdditiveSchemaMap`
helper that also DRYs the existing inbox/state merges.
- Entity projector + entity-types router accept the new field.
- Tests cover happy path, schema mismatch, undeclared collection,
reserved-name rejection, invalid name, non-object value,
stopped-entity, and route-layer delegation.
Runtime: `createEntityTimelineQuery` accepts an optional `customSource`
query-builder branch shaped to `EntityTimelineCustomRow`
(`{ collection, key, order, value }`) and splices it into the same
unionAll/orderBy pipeline as the built-in timeline collections, so
consumers project custom collections into one ordered live query.
`EntityDefinition.customCollectionSchemas` lets typed entity
definitions declare schemas that `create-handler.ts` forwards into the
type registration. `BUILT_IN_COLLECTION_TYPES` is exported as the
single source of truth for the reserved set.
Horton and worker entity types declare the `comment` schema from a
shared local module (`packages/agents/src/agents/comment-schema.ts`)
to opt their entities in to comments.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hema Renders session comments alongside the existing timeline rows and only exposes the comment surface on entities whose type declared a `comment` custom collection schema. Other entity types stay comment-free — the server would reject the writes anyway, so we just hide the surface. - `useEntityTimeline` registers the `comments` collection in the StreamDB customState, composes a `customSource` query branch shaped to the runtime's `EntityTimelineCustomRow` envelope, and hands it to `createEntityTimelineQuery` — one live query, no client-side merge. A small `projectRow` reshapes the runtime's generic `custom` variant back into a UI-friendly `comment` variant so renderers keep using `if (row.comment)`. - `lib/comments.ts` defines `CommentTarget`, `CommentSnapshot`, `CommentRow`, `EntityTimelineCommentRow`, the UI-facing `EntityTimelineQueryRow` augmentation, and the optimistic `createSendCommentAction` that posts to `POST .../collections/comment`. - `CommentBubble`, `EntityTimeline`, `MessageInput`, `ChatView`, `SplitMenu`, and adjacent components render comment bubbles, reply affordances on user messages, assistant responses, tool calls, and inline events, the reply preview iMessage-style with cross-tile focus handoff, a "comments only" tile view, and the `comments=hidden` view param. - ChatView derives `commentsEnabled` from the matching entity type's `custom_collection_schemas.comment` and AND's it into `showComments`, so entities whose type didn't opt in get the chat surface without any comment affordances. - `registerViews` gates the comments-only tile view registration on the same predicate via `isAvailable`. - `ElectricEntity` / `ElectricEntityType` zod schemas accept the new `custom_collection_schemas` field flowing through the entity stream. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`EntityDefinition.customCollectionSchemas` is loosened from `Record<string, StandardJSONSchemaV1>` to `Record<string, StandardSchemaV1 | Readonly<Record<string, unknown>>>` — the strict type was aspirational (no caller in the repo populates the parallel `inboxSchemas` / `stateSchemas` fields via the typed API; the only real callers send raw JSON Schema over the wire), and the runtime's `toJsonSchema()` already accepts Standard Schemas (Zod) and raw JSON Schema objects both. The looser type is honest about what works. With that in place, `comment-schema.ts` becomes a Zod definition (~50 lines, properties inferred at the type level) instead of a hand-rolled JSON Schema wrapped in a `jsonSchema()` envelope. The runtime continues to convert via `.toJSONSchema()` at registration time. The previously added `jsonSchema()` helper in `agents-runtime` and its re-export are dropped — no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
balegas
force-pushed
the
custom-collections
branch
from
744274a to
4fc0b2e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a generic mechanism for custom collections on entity streams and wires session comments on top of it. Comments are an opt-in feature on
hortonandworkerentity types; other entity types stay comment-free. Replaces the bespokePOST /commentsroute + built-incommentsruntime collection from #4529 with a layered design that's reusable for any other custom collection (e.g. samwillis's planned yjs doc comments).How it works
Declaration. Entity types declare the custom collections they accept via
custom_collection_schemas: { <name>: <JSON Schema> }.EntityDefinition.customCollectionSchemascarries this through the typed registration API; the runtime forwards it viacreate-handler.tsto the server, where it's persisted onentity_types(new JSONB column, migration 0015) and snapshotted onto each entity at spawn time.hortonandworkerdeclare acommentschema; the schema itself lives inpackages/agents/src/agents/comment-schema.tsas a Zod definition.Write path. A single generic endpoint,
POST /_electric/entities/:type/:id/collections/:name, accepts opaque JSON, looks up the schema declared for<name>on the entity's type viagetEffectiveSchemas, and rejects writes (422) when no schema is declared or the value doesn't validate. Reserved built-in collection names (BUILT_IN_COLLECTION_TYPES, exported from@electric-ax/agents-runtime) are rejected too so the runtime stays the sole writer of agent-managed collections.amendSchemassupports additive amendments of custom collection schemas via a sharedmergeAdditiveSchemaMaphelper that also DRYs the existing inbox/state merges.Read path.
createEntityTimelineQuerytakes an optionalcustomSourcequery-builder branch shaped toEntityTimelineCustomRow({ collection, key, order, value }) and splices it into the same unionAll/orderBy pipeline as the built-in timeline collections. Consumers get one ordered live query instead of merging a second source client-side — no two-source ordering race. Multi-collection callers can union their sources client-side and use thecollectionfield as a discriminator.UI.
useEntityTimelineregisters acommentscollection in the StreamDB customState (with a passthrough Standard Schema for the runtime row type), composes acustomSourcefor it, hands it tocreateEntityTimelineQuery, and reshapes envelope rows into a UI-friendlycommentvariant via a smallprojectRowthat spreadsrow.custom.valueand overrideskey/order. Comment surface — bubbles, reply affordances on user messages, agent responses, tool calls, and inline events, the comments-only tile view, and the reply composer — is all gated on whether the entity's type opted in to acommentschema.ElectricEntity/ElectricEntityTypecarrycustom_collection_schemasthrough their zod schemas, and theentity_typesElectric shape's column allow-list (server-utils.ts) was updated to expose the new column to UI clients.Notable design points
EntityDefinition.customCollectionSchemasis typed asRecord<string, StandardSchemaV1 | Readonly<Record<string, unknown>>>rather than the strictRecord<string, StandardJSONSchemaV1>used byinboxSchemas/stateSchemas. The runtime'stoJsonSchema()accepts Standard Schemas (Zod), raw JSON Schema objects, and anything with a~standard.jsonSchemaconverter — so the looser union is honest about what works in practice. The strict variant has no callers in the repo (the conformance tests send raw JSON Schema over the wire, going around the typed API).{ collection, key, order, value }) keeps the caller's.select(...)to a singlevalue: rowline — adding a field toCommentRowlater only needs to land inprojectRow, not in two places. The runtime guaranteeskeyandorder(the minimum it needs for the union and orderBy); the rest rides throughvalue.Test plan
pnpm --filter @electric-ax/agents-runtime run typecheck && run test— 57 timeline tests incl. 2 new custom-source testspnpm --filter @electric-ax/agents-server run typecheck && run test— 55 targeted tests incl. write-validation + route delegation (happy path, schema mismatch, undeclared collection, reserved-name rejection, invalid name, non-object value, stopped-entity, route-layer delegation)pnpm --filter @electric-ax/agents-server-ui run typecheck && run test— 93 tests incl. 3 new comments-action tests, 3 ChatView tests, 1 InlineEventCard testpnpm --filter @electric-ax/agents run typecheckcustom_collection_schemas.commentdeclared, the reply icons and comments-only view are correctly hidden.Notes
0015_entity_custom_collection_schemas.sqladds a nullable JSONB column on bothentity_typesandentities— additive, no backfill needed.entity_typesshape's column allow-list atpackages/agents-server/src/utils/server-utils.ts:138was updated to includecustom_collection_schemas— without this the UI gate would silently fail closed.BUILT_IN_COLLECTION_TYPESand theEntityTimelineCustomRow/EntityTimelineCustomSourcetypes as new public surface.