Link Search Menu Expand Document Documentation Menu
Documentation

Auto-follow for cross-cluster replication

Auto-follow lets you automatically replicate indexes created on the leader cluster based on matching patterns. When you create an index on the leader cluster with a name that matches a specified pattern (for example, index-01*), a corresponding follower index is automatically created on the follower cluster.

You can configure multiple replication rules for a single cluster. The patterns only support wildcard matching.

Prerequisites

You need to set up a cross-cluster connection between two clusters before you can enable auto-follow.

Permissions

If the Security plugin is enabled, make sure that non-admin users are mapped to the appropriate permissions so they can perform replication actions. For index and cluster-level permissions requirements, see Cross-cluster replication permissions.

Get started with auto-follow

Replication rules are a collection of patterns that you create against a single follower cluster. When you create a replication rule, it starts by automatically replicating any existing indexes that match the pattern. It will then continue to replicate any new indexes that you create that match the pattern.

Create a replication rule on the follower cluster:

curl -XPOST -k -H 'Content-Type: application/json' -u 'admin:<custom-admin-password>' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "my-connection-alias",
   "name": "my-replication-rule",
   "pattern": "movies*",
   "use_roles":{
      "leader_cluster_role": "all_access",
      "follower_cluster_role": "all_access"
   }
}'

If the Security plugin is disabled, you can leave out the use_roles parameter. If it’s enabled, however, you need to specify the leader and follower cluster roles that OpenSearch uses to authenticate requests. This example uses all_access for simplicity, but we recommend creating a replication user on each cluster and mapping it accordingly.

To test the rule, create a matching index on the leader cluster:

curl -XPUT -k -H 'Content-Type: application/json' -u 'admin:<custom-admin-password>' 'https://localhost:9201/movies-0001?pretty'

Confirm that the replica appears on the follower cluster:

curl -XGET -u 'admin:<custom-admin-password>' -k 'https://localhost:9200/_cat/indices?v'

It might take several seconds for the index to appear.

health status index        uuid                     pri rep docs.count docs.deleted store.size pri.store.size
yellow open   movies-0001  kHOxYYHxRMeszLjTD9rvSQ     1   1          0            0       208b           208b

Customize follower index names

By default, auto-follow gives each follower index the same name as its leader index. To apply a different naming convention to all follower indexes that a replication rule creates, specify the follower_index_pattern parameter. Custom names prevent name collisions when the follower cluster already contains a local index with the same name as a leader index or when you replicate indexes from multiple leader clusters.

The follower_index_pattern value can contain static text and the {{leader_index}} placeholder. At replication time, OpenSearch replaces the placeholder with the name of the matched leader index, so you can add a prefix, a suffix, or both.

The following request creates a replication rule that appends a -replica suffix to each follower index name:

curl -XPOST -k -H 'Content-Type: application/json' -u 'admin:<custom-admin-password>' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "my-connection-alias",
   "name": "my-replication-rule",
   "pattern": "movies*",
   "follower_index_pattern": "{{leader_index}}-replica",
   "use_roles":{
      "leader_cluster_role": "all_access",
      "follower_cluster_role": "all_access"
   }
}'

Using this rule, OpenSearch replicates a leader index named movies-2025 to a follower index named movies-2025-replica.

Retrieve replication rules

To retrieve a list of existing replication rules that are configured on a cluster, send the following request:

curl -XGET -u 'admin:<custom-admin-password>' -k 'https://localhost:9200/_plugins/_replication/autofollow_stats'

The response contains the replication rules and their statistics:

{
   "num_success_start_replication": 1,
   "num_failed_start_replication": 0,
   "num_failed_leader_calls": 0,
   "failed_indices":[
      
   ],
   "autofollow_stats":[
      {
         "name":"my-replication-rule",
         "pattern":"movies*",
         "num_success_start_replication": 1,
         "num_failed_start_replication": 0,
         "num_failed_leader_calls": 0,
         "failed_indices":[
            
         ]
      }
   ]
}

Delete a replication rule

To delete a replication rule, send the following request to the follower cluster:

curl -XDELETE -k -H 'Content-Type: application/json' -u 'admin:<custom-admin-password>' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "my-connection-alias",
   "name": "my-replication-rule"
}'

When you delete a replication rule, OpenSearch stops replicating new indexes that match the pattern, but existing indexes that the rule previously created remain read-only and continue to replicate. If you need to stop existing replication activity and open the indexes up for writes, use the stop replication API operation.

350 characters left

Have a question? .

Want to contribute? or .