Create or Update Snapshot Repository API

Introduced 1.0

Registers a new snapshot repository or updates an existing one.

Snapshot repositories can be of the following types:

• File system (fs): For more information about creating an fs repository, see Register repository shared file system.

• Amazon Simple Storage Service (Amazon S3) bucket (s3): For more information about creating an s3 repository, see Register repository Amazon S3.

• Hadoop Distributed File System (HDFS) (hdfs): For more information about creating an hdfs repository, see Register an HDFS repository.

For more information about creating a repository, see Register repository.

To update an existing repository, send a request containing the repository’s full definition. The new settings replace all existing settings, so any setting that you don’t include in the request is reset to its default value.

Because the snapshot format can change between major versions, we recommend registering a separate repository for each major version of OpenSearch. For more information, see Sharing a repository between clusters.

Endpoints

POST /_snapshot/{repository}
PUT /_snapshot/{repository}

Path parameters

The following table lists the available path parameters.

Query parameters

The following table lists the available query parameters. All query parameters are optional.

Request body fields

The following table lists the available request body fields.

Common parameters

The following table lists parameters that can be used with both the fs and s3 repositories.

shard_path_type values

The following values are supported in the shard_path_type setting:

• FIXED: Keeps the path structure in the existing hierarchical manner, such as <ROOT>/<BASE_PATH>/indices/<index-id>/0/<SHARD_BLOBS>.
• HASHED_PREFIX: Prepends a hashed prefix at the start of the path for each unique shard ID, for example, <ROOT>/<HASH-OF-INDEX-ID-AND-SHARD-ID>/<BASE_PATH>/indices/<index-id>/0/<SHARD_BLOBS>.
• HASHED_INFIX: Appends a hashed prefix after the base path for each unique shard ID, for example, <ROOT>/<BASE-PATH>/<HASH-OF-INDEX-ID-AND-SHARD-ID>/indices/<index-id>/0/<SHARD_BLOBS>. The hash method used is FNV_1A_COMPOSITE_1, which uses the FNV1a hash function and generates a custom-encoded 64-bit hash value that scales well with most remote store options. FNV1a takes the most significant 6 bits to create a URL-safe Base64 character and the next 14 bits to create a binary string.

fs repository

The following table lists the request fields for fs repositories.

s3 repository

The following table lists the request fields for s3 repositories.

The server_side_encryption setting was removed in OpenSearch 3.1.0. S3 applies server-side encryption as the base level of encryption for all S3 buckets. Because this encryption cannot be disabled, the repository setting had no effect. For more information, see Protecting data with server-side encryption.

s3_async_client_type

s3_async_client_type applies to a single repository, so set it in the repository’s settings block. The repository-s3 plugin also registers s3_async_client_type as a node setting, so a node starts successfully with the setting in opensearch.yml and the Nodes Info API returns it, but the plugin reads the value only from the repository definition. A repository that does not set s3_async_client_type uses crt, even when opensearch.yml specifies netty.

If the AWS Common Runtime native library is not available on the node’s platform, the plugin logs a warning and uses the Netty client.

To confirm which client a repository uses, enable DEBUG logging for the org.opensearch.repositories.s3.S3AsyncService logger and look for the S3 Http client type message.

hdfs repository

The following table lists the request fields for hdfs repositories.

url repository

A url repository is read-only: you can list and restore snapshots from it, but you can’t create snapshots in it. Use a url repository to give a cluster read-only access to snapshots that another cluster writes to a shared file system repository.

The following table lists the request fields for url repositories.

Example requests

The following examples demonstrate how to register different repository types.

fs

The following example registers an fs repository using the local directory /mnt/snapshots as location:

PUT /_snapshot/my-fs-repository
{
  "type": "fs",
  "settings": {
    "location": "/mnt/snapshots"
  }
}

response = client.snapshot.create_repository(
  repository = "my-fs-repository",
  body =   {
    "type": "fs",
    "settings": {
      "location": "/mnt/snapshots"
    }
  }
)

s3

The following request registers a new S3 repository called my-opensearch-repo in an existing bucket called my-open-search-bucket. Because the request sets base_path, snapshots are stored in the my/snapshot/directory path within the bucket:

PUT /_snapshot/my-opensearch-repo
{
  "type": "s3",
  "settings": {
    "bucket": "my-open-search-bucket",
    "base_path": "my/snapshot/directory"
  }
}

response = client.snapshot.create_repository(
  repository = "my-opensearch-repo",
  body =   {
    "type": "s3",
    "settings": {
      "bucket": "my-open-search-bucket",
      "base_path": "my/snapshot/directory"
    }
  }
)

The following request registers a new S3 repository called my-opensearch-repo in an existing bucket called my-open-search-bucket. Because the request sets base_path, snapshots are stored in the my/snapshot/directory path within the bucket. Additionally, this repository is configured to use SSE-KMS, and the expected bucket owner AWS account ID is 123456789000:

PUT /_snapshot/my-opensearch-repo
{
  "type": "s3",
  "settings": {
    "bucket": "my-open-search-bucket",
    "base_path": "my/snapshot/directory",
    "server_side_encryption_type": "aws:kms",
    "server_side_encryption_kms_key_id": "arn:aws:kms:us-east-1:123456789000:key/kms-key-id",
    "server_side_encryption_encryption_context": "{\"additional-enc-ctx\": \"sample-context\"}",
    "expected_bucket_owner": "123456789000"
  }
}

response = client.snapshot.create_repository(
  repository = "my-opensearch-repo",
  body =   {
    "type": "s3",
    "settings": {
      "bucket": "my-open-search-bucket",
      "base_path": "my/snapshot/directory",
      "server_side_encryption_type": "aws:kms",
      "server_side_encryption_kms_key_id": "arn:aws:kms:us-east-1:123456789000:key/kms-key-id",
      "server_side_encryption_encryption_context": "{\"additional-enc-ctx\": \"sample-context\"}",
      "expected_bucket_owner": "123456789000"
    }
  }
)

hdfs

The following request registers a new HDFS repository using the HDFS URI hdfs://namenode:8020 and the HDFS file system path /opensearch/snapshots:

PUT /_snapshot/my-hdfs-repository
{
  "type": "hdfs",
  "settings": {
    "uri": "hdfs://namenode:8020",
    "path": "/opensearch/snapshots"
  }
}

response = client.snapshot.create_repository(
  repository = "my-hdfs-repository",
  body =   {
    "type": "hdfs",
    "settings": {
      "uri": "hdfs://namenode:8020",
      "path": "/opensearch/snapshots"
    }
  }
)

url

The following request registers a read-only url repository for the shared file system location /mnt/snapshots:

PUT /_snapshot/my-url-repository
{
  "type": "url",
  "settings": {
    "url": "file:/mnt/snapshots"
  }
}

response = client.snapshot.create_repository(
  repository = "my-url-repository",
  body =   {
    "type": "url",
    "settings": {
      "url": "file:/mnt/snapshots"
    }
  }
)

Example response

Upon success, the following JSON object is returned:

{
  "acknowledged": true
}

To verify that the repository was registered, use the Get Snapshot Repository API, passing the repository name as the repository path parameter.

Required permissions

If you use the Security plugin, make sure you have the appropriate permissions: cluster:admin/repository/put.