Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 42 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,15 @@ It's all in one database, with vector + BM25 + full-text + trigram + PostGIS sea
- **BM25** via `pg_textsearch` — statistical relevance ranking, not just similarity.
- **Weighted FTS** — `A` / `B` / `C` weights per field so `name > headline > bio` naturally.
- **Trigram fuzzy matching** for typo-tolerant name search.
- **PostGIS spatial** on contacts, events, venues, places, memories, trips — "find memories within 5km of here" works out of the box.
- **PostGIS spatial** on contacts, events, venues, places, memories, trips — `Point` geography columns auto-backed by a `GIST` index. Scalar filters (`bboxIntersects2D`, `isNull`) ship on every geom column today; single-table radius queries are a schema decision away.
- **Cross-table spatial relations (`@spatialRelation`)** — FK-less spatial joins exposed as named `where:` filters. The blueprint ships 5 out of the box (see [`packages/provision/src/schemas/spatial-relations.ts`](packages/provision/src/schemas/spatial-relations.ts)):
- `memory.nearbyPlaces` — memories within N metres of any `place` (5 km default)
- `memory.nearbyContacts` — memories within N metres of any `contact` (2 km default)
- `trip.nearbyVenues` — trips whose `destination_geo` is within N metres of a `venue` (1 km default)
- `event.nearbyVenues` — events within N metres of a `venue`, no FK needed (500 m default)
- `memory.nearbyMemories` — self-join, "what else happened near this memory?" (1 km default)

All 5 use `st_dwithin` with a per-query `distance` param, so radius is a query-time input not a schema constant. Each renders server-side as an `EXISTS (… ST_DWithin(geo_a, geo_b, $distance) …)` subquery — zero GeoJSON on the wire.
- **Chunked long-doc retrieval** on contacts and notes.

### 🌍 World model (context the agent needs to actually help you)
Expand Down Expand Up @@ -276,9 +284,41 @@ const results = await db.contact
select: { id: true, firstName: true, searchScore: true },
})
.execute();

// Unified search + cross-table PostGIS spatial filter, composed in one where:
// "Memories whose content semantically matches 'conference keynote' AND are
// within 5 km of a place tagged as a market — return each with its
// relevance score and stored GeoJSON point, all in one round-trip."
const ranked = await db.memory
.findMany({
where: {
unifiedSearch: 'conference keynote retrieval',
nearbyPlaces: {
distance: 5000, // metres — columns are geography so distance is metric
some: { category: { equalTo: 'market' } },
},
},
first: 10,
select: {
id: true,
title: true,
searchScore: true,
locationGeo: { select: { geojson: true, srid: true } },
},
})
.execute();
```

Full ORM reference in the [`orm-default` skill](skills/orm-default/SKILL.md); integration tests in [`packages/integration-tests/__tests__/orm.test.ts`](packages/integration-tests/__tests__/orm.test.ts).
The server compiles that into a single SQL statement: a `unified_search(...)` ranked CTE joined against an `EXISTS (… ST_DWithin(memory.location_geo, place.location_geo, 5000) …)` subquery. No GeoJSON goes over the wire on the spatial side, and the text-search score comes back as `searchScore`.

That exact combined shape is exercised end-to-end by an integration test: [`packages/agentic-db/__tests__/unified-spatial-combined.test.ts`](packages/agentic-db/__tests__/unified-spatial-combined.test.ts). It boots a real deploy of the agentic-db pgpm package, seeds three memories and two market-category places at known coordinates, runs the same `memory.findMany({ where: { unifiedSearch, nearbyPlaces } })` call through the generated SDK, and asserts that only the positive-match memory comes back with a non-null `searchScore`.

Supporting single-axis coverage lives alongside:

- **Spatial-only** — `nearbyPlaces` / `nearbyContacts` / `nearbyVenues` / `nearbyMemories` are each exercised in [`packages/integration-tests/__tests__/orm.test.ts`](packages/integration-tests/__tests__/orm.test.ts) under the `RelationSpatial via ORM` describe block (all 5 relations declared in the blueprint).
- **Unified-search only** — `unifiedSearch` + `searchScore` ranking behavior is covered by [`packages/agentic-db/__tests__/rag-unified-search.test.ts`](packages/agentic-db/__tests__/rag-unified-search.test.ts) against pre-baked `nomic-embed-text` fixtures (no Ollama required).

Full ORM reference in the [`orm-default` skill](skills/orm-default/SKILL.md).

## Packages

Expand Down
44 changes: 42 additions & 2 deletions packages/agentic-db/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,15 @@ It's all in one database, with vector + BM25 + full-text + trigram + PostGIS sea
- **BM25** via `pg_textsearch` — statistical relevance ranking, not just similarity.
- **Weighted FTS** — `A` / `B` / `C` weights per field so `name > headline > bio` naturally.
- **Trigram fuzzy matching** for typo-tolerant name search.
- **PostGIS spatial** on contacts, events, venues, places, memories, trips — "find memories within 5km of here" works out of the box.
- **PostGIS spatial** on contacts, events, venues, places, memories, trips — `Point` geography columns auto-backed by a `GIST` index. Scalar filters (`bboxIntersects2D`, `isNull`) ship on every geom column today; single-table radius queries are a schema decision away.
- **Cross-table spatial relations (`@spatialRelation`)** — FK-less spatial joins exposed as named `where:` filters. The blueprint ships 5 out of the box (see [`packages/provision/src/schemas/spatial-relations.ts`](packages/provision/src/schemas/spatial-relations.ts)):
- `memory.nearbyPlaces` — memories within N metres of any `place` (5 km default)
- `memory.nearbyContacts` — memories within N metres of any `contact` (2 km default)
- `trip.nearbyVenues` — trips whose `destination_geo` is within N metres of a `venue` (1 km default)
- `event.nearbyVenues` — events within N metres of a `venue`, no FK needed (500 m default)
- `memory.nearbyMemories` — self-join, "what else happened near this memory?" (1 km default)

All 5 use `st_dwithin` with a per-query `distance` param, so radius is a query-time input not a schema constant. Each renders server-side as an `EXISTS (… ST_DWithin(geo_a, geo_b, $distance) …)` subquery — zero GeoJSON on the wire.
- **Chunked long-doc retrieval** on contacts and notes.

### 🌍 World model (context the agent needs to actually help you)
Expand Down Expand Up @@ -276,9 +284,41 @@ const results = await db.contact
select: { id: true, firstName: true, searchScore: true },
})
.execute();

// Unified search + cross-table PostGIS spatial filter, composed in one where:
// "Memories whose content semantically matches 'conference keynote' AND are
// within 5 km of a place tagged as a market — return each with its
// relevance score and stored GeoJSON point, all in one round-trip."
const ranked = await db.memory
.findMany({
where: {
unifiedSearch: 'conference keynote retrieval',
nearbyPlaces: {
distance: 5000, // metres — columns are geography so distance is metric
some: { category: { equalTo: 'market' } },
},
},
first: 10,
select: {
id: true,
title: true,
searchScore: true,
locationGeo: { select: { geojson: true, srid: true } },
},
})
.execute();
```

Full ORM reference in the [`orm-default` skill](skills/orm-default/SKILL.md); integration tests in [`packages/integration-tests/__tests__/orm.test.ts`](packages/integration-tests/__tests__/orm.test.ts).
The server compiles that into a single SQL statement: a `unified_search(...)` ranked CTE joined against an `EXISTS (… ST_DWithin(memory.location_geo, place.location_geo, 5000) …)` subquery. No GeoJSON goes over the wire on the spatial side, and the text-search score comes back as `searchScore`.

That exact combined shape is exercised end-to-end by an integration test: [`packages/agentic-db/__tests__/unified-spatial-combined.test.ts`](__tests__/unified-spatial-combined.test.ts). It boots a real deploy of the agentic-db pgpm package, seeds three memories and two market-category places at known coordinates, runs the same `memory.findMany({ where: { unifiedSearch, nearbyPlaces } })` call through the generated SDK, and asserts that only the positive-match memory comes back with a non-null `searchScore`.

Supporting single-axis coverage lives alongside:

- **Spatial-only** — `nearbyPlaces` / `nearbyContacts` / `nearbyVenues` / `nearbyMemories` are each exercised in [`packages/integration-tests/__tests__/orm.test.ts`](../integration-tests/__tests__/orm.test.ts) under the `RelationSpatial via ORM` describe block (all 5 relations declared in the blueprint).
- **Unified-search only** — `unifiedSearch` + `searchScore` ranking behavior is covered by [`packages/agentic-db/__tests__/rag-unified-search.test.ts`](__tests__/rag-unified-search.test.ts) against pre-baked `nomic-embed-text` fixtures (no Ollama required).

Full ORM reference in the [`orm-default` skill](../../skills/orm-default/SKILL.md).

## Packages

Expand Down
203 changes: 203 additions & 0 deletions packages/agentic-db/__tests__/unified-spatial-combined.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
/**
* Combined Unified Search + RelationSpatial test — ORM.
*
* Proves the exact query shape documented in the root README's
* "Use the SDK (ORM)" section: a single `where:` clause that
* composes the unified-search filter (`unifiedSearch: '...'`) with
* a cross-table PostGIS spatial relation (`nearbyPlaces: { distance,
* some: { ... } }`).
*
* Harness: `@constructive-io/graphql-test`'s `getConnections()` boots
* the full `ConstructivePreset` plugin stack — which includes both
* `UnifiedSearchPreset` (graphile-search) AND
* `PostgisSpatialRelationsPlugin` — against a real deploy of the
* agentic-db pgpm package. Same stack that `cnc server` serves in
* production, so this test exercises the documented query shape
* end-to-end through the generated SDK.
*
* Fixture data is seeded via raw SQL at known coordinates so the
* distance half is deterministic and the text half has a clean
* positive + two negatives.
*/
jest.setTimeout(300000);
process.env.LOG_SCOPE = '@constructive-io/graphql-test';

import { getConnections, GraphQLTestAdapter } from '@constructive-io/graphql-test';
import type { GraphQLQueryFn } from '@constructive-io/graphql-test';
import { createClient } from '@agentic-db/sdk';
import {
createAppJobsStub,
grantAnonymousAccess,
} from '../test-utils/helpers';

const SCHEMAS = ['agentic_db_app_public'];

// Deterministic UUIDs so the assertions can name-check matches / negatives.
const AGENT_ID = '00000000-0000-0000-0000-0000000000a1';
const MEMORY_SF = '00000000-0000-0000-0000-0000000000b1';
const MEMORY_OAKLAND = '00000000-0000-0000-0000-0000000000b2';
const MEMORY_NYC = '00000000-0000-0000-0000-0000000000b3';
const PLACE_FERRY = '00000000-0000-0000-0000-0000000000c1';
const PLACE_TOKYO = '00000000-0000-0000-0000-0000000000c2';

let db: any;
let pg: any;
let query: GraphQLQueryFn;
let teardown: () => Promise<void>;

beforeAll(async () => {
const connections = await getConnections({
schemas: SCHEMAS,
authRole: 'anonymous',
});
({ db, pg, query, teardown } = connections);

await grantAnonymousAccess(pg);
await createAppJobsStub(pg);
});

afterAll(async () => {
if (teardown) await teardown();
});

// Each test runs in its own transaction (begun here, rolled back
// after the test). Seeding happens inside the transaction so the
// ORM query in `it(...)` sees the inserted rows.
beforeEach(() => db.beforeEach());
afterEach(() => db.afterEach());

describe('Unified search + RelationSpatial composition via ORM', () => {
beforeEach(async () => {
// Minimal agent row so memories have a valid agent_id FK.
await pg.query(
`
INSERT INTO agentic_db_app_public.agents (id, name)
VALUES ($1, 'test-agent')
ON CONFLICT (id) DO NOTHING
`,
[AGENT_ID]
);

// Memories — raw SQL so location_geo can be set as a geography
// Point in one statement. Title/content chosen so only MEMORY_SF
// has meaningful text overlap with the query term
// "Ferry Building coffee".
await pg.query(
`
INSERT INTO agentic_db_app_public.memories
(id, agent_id, title, content, location_geo)
VALUES
(
$1, $4,
'Ferry Building keynote recap',
'Met a collaborator over coffee near the Ferry Building after the retrieval keynote.',
ST_SetSRID(ST_MakePoint(-122.4194, 37.7749), 4326)::geography
),
(
$2, $4,
'Oakland lunch',
'Reviewed an unrelated benchmark over lunch.',
ST_SetSRID(ST_MakePoint(-122.2712, 37.8044), 4326)::geography
),
(
$3, $4,
'NYC meetup',
'Caught up with the east-coast team; had pasta.',
ST_SetSRID(ST_MakePoint(-74.0060, 40.7128), 4326)::geography
)
ON CONFLICT (id) DO NOTHING
`,
[MEMORY_SF, MEMORY_OAKLAND, MEMORY_NYC, AGENT_ID]
);

// Places — one matches the `category='market'` predicate and is
// ~200 m from MEMORY_SF, ~13 km from MEMORY_OAKLAND, ~4100 km
// from MEMORY_NYC. The Tokyo row exists to make sure the
// spatial predicate actually filters (no memory is within 5 km
// of Tokyo).
await pg.query(
`
INSERT INTO agentic_db_app_public.places
(id, name, category, location_geo)
VALUES
(
$1,
'Ferry Building Marketplace',
'market',
ST_SetSRID(ST_MakePoint(-122.3937, 37.7956), 4326)::geography
),
(
$2,
'Tsukiji Outer Market',
'market',
ST_SetSRID(ST_MakePoint(139.7700, 35.6655), 4326)::geography
)
ON CONFLICT (id) DO NOTHING
`,
[PLACE_FERRY, PLACE_TOKYO]
);

await db.publish();
});

it('memory.findMany(unifiedSearch + nearbyPlaces): composes text + spatial in one where', async () => {
const sdk = createClient({ adapter: new GraphQLTestAdapter(query) });

const result = await sdk.memory
.findMany({
where: {
// Text half — matches on title/content via any of FTS,
// BM25 or trgm depending on what's configured on the
// underlying columns. MEMORY_SF contains both "Ferry
// Building" and "coffee", so it scores strongly.
unifiedSearch: 'Ferry Building coffee',
// Spatial half — @spatialRelation smart tag on
// memory.location_geo (declared in
// packages/provision/src/schemas/spatial-relations.ts)
// exposes `nearbyPlaces` on MemoryFilter. Body uses the
// plugin's `{ distance, some: { …PlaceFilter… } }` shape.
nearbyPlaces: {
distance: 5000,
some: { category: { equalTo: 'market' } },
},
},
first: 10,
select: {
id: true,
title: true,
searchScore: true,
},
})
.execute();

if (!result.ok) {
throw new Error(
`combined unifiedSearch+nearbyPlaces query failed: ${JSON.stringify(result.errors, null, 2)}`
);
}

const nodes = result.data.memories.nodes;
const ids = nodes.map((n: any) => n.id);

// MEMORY_SF passes BOTH halves: its text matches the query and
// it's within 5 km of the Ferry Building Marketplace (market).
expect(ids).toContain(MEMORY_SF);

// MEMORY_OAKLAND fails BOTH halves: text doesn't overlap, and
// it's ~13 km from any market-category place (Ferry Building is
// 13 km away; Tsukiji is across the Pacific).
expect(ids).not.toContain(MEMORY_OAKLAND);

// MEMORY_NYC fails BOTH halves: "pasta" doesn't overlap the
// query, and NYC is ~4100 km from the nearest market.
expect(ids).not.toContain(MEMORY_NYC);

// The unified-search plugin populates `searchScore` as a
// 0..1 blended relevance signal when any text algorithm fires.
const sf = nodes.find((n: any) => n.id === MEMORY_SF);
expect(sf).toBeDefined();
expect(typeof sf.searchScore).toBe('number');
expect(sf.searchScore).toBeGreaterThan(0);
expect(sf.searchScore).toBeLessThanOrEqual(1);
});
});
2 changes: 1 addition & 1 deletion packages/agentic-db/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
"@agentic-db/sdk": "workspace:*",
"@agentic-kit/ollama": "^1.0.3",
"@constructive-io/graphql-test": "^4.9.10",
"graphile-settings": "4.18.5",
"graphile-settings": "4.21.0",
"pgsql-test": "^4.7.6"
}
}
Loading
Loading