From 41dc5ee86f8a0536519b1b03255bde9e0403405c Mon Sep 17 00:00:00 2001 From: Shwetha Rao Date: Thu, 7 May 2026 19:24:04 +0530 Subject: [PATCH] draft-1 --- .../xdcr-filtering.adoc | 45 +++++++++++++++++++ .../pages/xdcr-filtering-expressions.adoc | 34 ++++++++++++++ 2 files changed, 79 insertions(+) diff --git a/modules/learn/pages/clusters-and-availability/xdcr-filtering.adoc b/modules/learn/pages/clusters-and-availability/xdcr-filtering.adoc index 4ac1d3a44a..5603f1840f 100644 --- a/modules/learn/pages/clusters-and-availability/xdcr-filtering.adoc +++ b/modules/learn/pages/clusters-and-availability/xdcr-filtering.adoc @@ -324,3 +324,48 @@ Note that before and during migration, both the xref:learn:services-and-indexes/ For information on configuring deletion filters with the UI, see xref:manage:manage-xdcr/filter-xdcr-replication.adoc#deletion-filters[Deletion Filters]; with the CLI, see xref:cli:cbcli/couchbase-cli-xdcr-replicate.adoc[xdcr-replicate]; with the REST API, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating XDCR Replications]. + +== Applying Expressions to Tombstones + +When a document is deleted or expires, Couchbase creates a *tombstone*[cite: 1]. A tombstone contains only the **document key (ID)** and system metadata (xattrs); it does not contain the document body[cite: 1]. + +Because tombstones lack a body, standard filter expressions that reference document fields cannot be evaluated and are bypassed, allowing the deletion to replicate to all targets[cite: 1]. The new option, *Apply deletions (tombstones) to filter expressions using doc keys only*, allows you to filter these deletions if your expression relies exclusively on the document ID[cite: 1]. + +=== Use Case: Non-Unique Global Keys +In certain architectures, document IDs might not be globally unique across all sources[cite: 1]. Without tombstone filtering, a deletion from Source A could inadvertently delete a document with the same ID on Target B, even if Source A was never intended to replicate to Target B[cite: 1]. Enabling this feature ensures that a deletion only replicates if the document ID matches the specified filter[cite: 1]. + +=== Requirements and Limitations + +To enable tombstone filtering, your replication must meet the following criteria: + +* **Active Filter Expression**: A filter expression must be defined for the replication[cite: 1]. +* **Key-Only Expressions**: The expression must reference *only* the document ID (e.g., using `META().id`)[cite: 1]. If the expression references document values or extended attributes (xattrs), this option cannot be enabled[cite: 1]. +* **Opt-in Basis**: This feature is disabled by default to maintain backward compatibility with earlier Server behavior[cite: 1]. +* **Enterprise Edition**: XDCR filtering features are available only in Couchbase Server Enterprise Edition[cite: 1]. + +.Expression Compatibility Reference +[cols="2,1", options="header"] +|=== +| Filter Expression | Can Enable Tombstone Filtering? +| `REGEXP_CONTAINS(META().id, "^ABC-")` | Yes[cite: 1] +| `REGEXP_CONTAINS(country, "France")` | No (Uses doc values)[cite: 1] +| `REGEXP_CONTAINS(META().id, "^abc") AND REGEXP_CONTAINS(country, "France")` | No (Mixed keys/values)[cite: 1] +|=== + +=== Interaction with Deletion Filters + +Existing XDCR deletion filters take precedence over the tombstone filtering option[cite: 1]. + +* **If "Do not replicate DELETE operations" is enabled**: Application-level deletes are blocked entirely[cite: 1]. The tombstone filtering option will only apply to document expirations (TTL)[cite: 1]. +* **If "Do not replicate document expirations" is enabled**: Expirations are blocked entirely[cite: 1]. The tombstone filtering option will only apply to application-level deletes[cite: 1]. +* **If both are enabled**: All deletions are blocked; the tombstone filtering option has no effect[cite: 1]. + +=== Configure Tombstone Filtering + +You can enable or update this setting via the Couchbase Server Web Console, REST API, or CLI[cite: 1]. + +==== Using the Web Console +. Go to **XDCR** and click **Add Replication** (or click **Edit** on an existing replication)[cite: 1]. +. Enable **Filter Replication** and enter your expression[cite: 1]. +. Under **Deletion Filters**, check the box labeled **Apply deletions (tombstones) to filter expressions using doc keys only**[cite: 1]. +. Click **Save Replication**[cite: 1]. \ No newline at end of file diff --git a/modules/xdcr-reference/pages/xdcr-filtering-expressions.adoc b/modules/xdcr-reference/pages/xdcr-filtering-expressions.adoc index a3e8833ed9..c0796b2793 100644 --- a/modules/xdcr-reference/pages/xdcr-filtering-expressions.adoc +++ b/modules/xdcr-reference/pages/xdcr-filtering-expressions.adoc @@ -703,3 +703,37 @@ The relationships between available expressions for XDCR Advanced Filtering are | ExistsClause | ( "EXISTS" "(" Field ")" ) |=== + +== Configure Tombstone Filtering + +You can enable or update this setting via the Couchbase Server Web Console, REST API, or CLI[cite: 1]. + +=== Using the REST API +Use the `filterDeletionsWithKey` parameter when creating or updating a replication[cite: 1]. + +[source,console] +---- +curl -X POST http://localhost:8091/controller/createReplication \ +-u Administrator:password \ +-d fromBucket="travel-sample" \ +-d toCluster="remote-cluster" \ +-d toBucket="backup" \ +-d filterExpression="REGEXP_CONTAINS(META().id, '^route_')" \ +-d filterDeletionsWithKey=true +---- + +=== Using the CLI +Use the `--filter-deletions-with-key` flag with the `xdcr-replicate` command[cite: 1]. + +[source,console] +---- +couchbase-cli xdcr-replicate -c 127.0.0.1:8091 -u Administrator -p password \ +--create --xdcr-from-bucket source --xdcr-to-bucket target \ +--xdcr-to-cluster remote-cluster \ +--filter-expression "REGEXP_CONTAINS(META().id, '^ABC-')" \ +--filter-deletions-with-key 1 +---- + +== See Also +* Refer to the file "PRD _ XDCR - option to apply deletions to filter expressions using doc key.docx" for detailed technical specifications and requirements[cite: 1]. +* xref:learn/clusters-and-availability/xdcr-filtering.adoc[XDCR Advanced Filtering] \ No newline at end of file