operations

The operations element contains a list of all available operations for specifying a schedule.

bulk

The bulk operation type allows you to run bulk requests as a task.

Usage

The following example shows a bulk operation type with a bulk-size of 5000 documents:

{
  "name": "index-append",
  "operation-type": "bulk",
  "bulk-size": 5000
}

Split documents among clients

When you have multiple clients, OpenSearch Benchmark splits each document based on the set number of clients. Having multiple clients parallelizes the bulk index operations but doesn’t preserve the ingestion order of each document. For example, if clients is set to 2, one client indexes the document starting from the beginning, while the other client indexes the document starting from the middle.

If there are multiple documents or corpora, OpenSearch Benchmark attempts to index all documents in parallel in two ways:

1. Each client starts at a different point in the corpus. For example, in a workload with 2 corpora and 5 clients, clients 1, 3, and 5 begin with the first corpus, whereas clients 2 and 4 start with the second corpus.
2. Each client is assigned to multiple documents. Client 1 starts with the first split of the first document of the first corpus. Then it moves to the first split of the first document of the second corpus, and so on.

Configuration options

Use the following options to customize the bulk operation.

Metadata

The bulk operation always returns the following metadata:

• index: The name of the affected index. If an index cannot be derived, it returns null.
• weight: An operation-agnostic representation of the bulk size, denoted by units.
• unit: The unit used to interpret weight.
• success: A Boolean indicating whether the bulk request succeeded.
• success-count: The number of successfully processed bulk items for the request. This value is determined when there are errors or when the bulk-size has been specified in the documents.
• error-count: The number of failed bulk items for the request.
• took: The value of the took property in the bulk response.

If detailed-results is true, the following metadata is returned:

• ops: A nested document with the operation name as its key, such as index, update, or delete, and various counts as values. item-count contains the total number of items for this key. Additionally, OpenSearch Benchmark returns a separate counter for each result, for example, a result for the number of created items or the number of deleted items.
• shards_histogram: An array of hashes, each of which has two keys. The item-count key contains the number of items to which a shard distribution applies. The shards key contains a hash with the actual distribution of total, successful, and failed shards.
• bulk-request-size-bytes: The total size of the bulk request body, in bytes.
• total-document-size-bytes: The total size of all documents within the bulk request body, in bytes.

create-index

The create-index operation runs the Create Index API. It supports the following two index creation modes:

• Creating all indexes specified in the workload’s indices section
• Creating one specific index defined within the operation itself

Usage

The following example creates all indexes defined in the indices section of the workload. It uses all of the index settings defined in the workload but overrides the number of shards:

{
  "name": "create-all-indices",
  "operation-type": "create-index",
  "settings": {
    "index.number_of_shards": 1
  },
  "request-params": {
    "wait_for_active_shards": "true"
  }
}

The following example creates a new index with all index settings specified in the operation body:

{
  "name": "create-an-index",
  "operation-type": "create-index",
  "index": "people",
  "body": {
    "settings": {
      "index.number_of_shards": 0
    },
    "mappings": {
      "docs": {
        "properties": {
          "name": {
            "type": "text"
          }
        }
      }
    }
  }
}

Configuration options

Use the following options when creating all indexes from the indices section of a workload.

Use the following options when creating a single index in the operation.

Metadata

The create-index operation returns the following metadata:

• weight: The number of indexes created by the operation.
• unit: Always ops, indicating the number of operations inside the workload.
• success: A Boolean indicating whether the operation has succeeded.

delete-index

The delete-index operation runs the Delete Index API. As with the create-index operation, you can delete all indexes found in the indices section of the workload or delete one or more indexes based on the string passed in the index setting.

Usage

The following example deletes all indexes found in the indices section of the workload:

{
  "name": "delete-all-indices",
  "operation-type": "delete-index"
}

The following example deletes all logs_* indexes:

{
  "name": "delete-logs",
  "operation-type": "delete-index",
  "index": "logs-*",
  "only-if-exists": false,
  "request-params": {
    "expand_wildcards": "all",
    "allow_no_indices": "true",
    "ignore_unavailable": "true"
  }
}

Configuration options

Use the following options when deleting all indexes indicated in the indices section of the workload.

Use the following options if you want to delete one or more indexes based on the pattern indicated in the index option.

Metadata

The delete-index operation returns the following metadata:

• weight: The number of indexes created by the operation.
• unit: Always ops, for the number of operations inside the workload.
• success: A Boolean indicating whether the operation has succeeded.

cluster-health

The cluster-health operation runs the Cluster Health API, which checks the cluster health status and returns the expected status according to the parameters set for request-params. If an unexpected cluster health status is returned, then the operation reports a failure. You can use the --on-error option in the OpenSearch Benchmark run command to control how OpenSearch Benchmark behaves when the health check fails.

Usage

The following example creates a cluster-health operation that checks for a green health status on any log-* indexes:

{
  "name": "check-cluster-green",
  "operation-type": "cluster-health",
  "index": "logs-*",
  "request-params": {
    "wait_for_status": "green",
    "wait_for_no_relocating_shards": "true"
  },
  "retry-until-success": true
}

Configuration options

Use the following options with the cluster-health operation.

Metadata

The cluster-health operation returns the following metadata:

• weight: The number of indexes the cluster-health operation assesses. Always 1, since the operation runs once per index.
• unit: Always ops, for the number of operations inside the workload.
• success: A Boolean indicating whether the operation has succeeded.
• cluster-status: The current cluster status.
• relocating-shards: The number of shards currently relocating to a different node.

refresh

The refresh operation runs the Refresh API. The operation returns no metadata.

Usage

The following example refreshes all logs-* indexes:

{
 "name": "refresh",
 "operation-type": "refresh",
 "index": "logs-*"
}

Configuration options

The refresh operation uses the following options.

search

The search operation runs the Search API, which you can use to run queries in OpenSearch Benchmark indexes.

Usage

The following example runs a match_all query inside the search operation:

{
  "name": "default",
  "operation-type": "search",
  "body": {
    "query": {
      "match_all": {}
    }
  },
  "request-params": {
    "_source_include": "some_field",
    "analyze_wildcard": "false"
  }
}

Configuration options

The search operation uses the following options.

Metadata

The following metadata is always returned:

• weight: The “weight” of an operation. Always 1 for regular queries and the number of retrieved pages for scroll queries.
• unit: The unit used to interpret weight, which is ops for regular queries and pages for scroll queries.
• success: A Boolean indicating whether the query has succeeded.

If detailed-results is set to true, the following metadata is also returned:

• hits: The total number of hits for the query.
• hits_relation: Whether the number of hits is accurate (eq) or a lower bound of the actual hit count (gte).
• timed_out: Whether the query has timed out. For scroll queries, this flag is true if the flag was true for any of the queries issued.
• took: The value of the took property in the query response. For scroll queries, the value is the sum of all took values in all query responses.

paginated-search

The paginated-search operation runs a sequence of paginated search requests using the search_after parameter. It accepts the same options as search plus pages, which controls how many pages to retrieve.

Configuration options

In addition to the options listed under search:

Metadata

In addition to the metadata returned by search:

• pages: The total number of pages retrieved.

scroll-search

The scroll-search operation runs a scroll search. It accepts the same options as search plus pages, which controls how many scroll pages to fetch.

Configuration options

In addition to the options listed under search:

Metadata

In addition to the metadata returned by search:

• pages: The total number of scroll pages retrieved.

force-merge

The force-merge operation runs the Force Merge API.

Usage

{
  "name": "force-merge",
  "operation-type": "force-merge",
  "index": "_all",
  "request-params": {
    "max_num_segments": 1
  }
}

Configuration options

This is an administrative operation. Metrics are not reported by default. Reporting can be forced by setting include-in-reporting to true.

index-stats

The index-stats operation runs the Index Stats API.

Usage

{
  "name": "index-stats",
  "operation-type": "index-stats",
  "index": "_all",
  "condition": {
    "path": "_all.total.merges.current",
    "expected-value": 0
  },
  "retry-until-success": true
}

Configuration options

node-stats

The node-stats operation runs the Nodes Stats API.

Usage

{
  "name": "node-stats",
  "operation-type": "node-stats"
}

Configuration options

vector-search

The vector-search operation runs k-NN vector search queries and optionally computes recall metrics by comparing results against ground truth neighbors.

Usage

{
  "name": "knn-search",
  "operation-type": "vector-search",
  "index": "vectors",
  "k": 100,
  "body": {
    "query": {
      "knn": {
        "embedding": {
          "vector": [0.1, 0.2, 0.3],
          "k": 100
        }
      }
    }
  }
}

Configuration options

Metadata

• weight: Always 1.
• unit: Always ops.
• success: Whether the search succeeded.
• recall@k: Recall at k (if ground truth neighbors are provided).
• recall@1: Recall at 1 (if ground truth neighbors are provided).

bulk-vector-data-set

The bulk-vector-data-set operation bulk-indexes vector data from HDF5 or BigANN dataset files. Supports individual document retry on partial failures.

Usage

{
  "name": "bulk-vectors",
  "operation-type": "bulk-vector-data-set",
  "bulk-size": 500,
  "index": "vectors"
}

Configuration options

put-pipeline

The put-pipeline operation creates or updates an ingest pipeline.

Usage

{
  "name": "define-pipeline",
  "operation-type": "put-pipeline",
  "id": "my-pipeline",
  "body": {
    "description": "My ingest pipeline",
    "processors": [
      {
        "set": {
          "field": "ingest_time",
          "value": "{{_ingest.timestamp}}"
        }
      }
    ]
  }
}

Configuration options

This is an administrative operation. Metrics are not reported by default.

delete-pipeline

The delete-pipeline operation deletes an ingest pipeline.

Configuration options

This is an administrative operation. Metrics are not reported by default.

create-search-pipeline

The create-search-pipeline operation creates a search pipeline.

Configuration options

This is an administrative operation. Metrics are not reported by default.

update-concurrent-segment-search-settings

The update-concurrent-segment-search-settings operation toggles the concurrent segment search cluster setting and, optionally, the maximum slice count. It updates search.concurrent_segment_search.enabled (and search.concurrent.max_slice_count when provided) as persistent cluster settings.

Configuration options

put-settings

The put-settings operation updates cluster settings.

Usage

{
  "name": "increase-watermarks",
  "operation-type": "put-settings",
  "body": {
    "transient": {
      "cluster.routing.allocation.disk.watermark.low": "95%"
    }
  }
}

Configuration options

This is an administrative operation. Metrics are not reported by default.

create-data-stream

The create-data-stream operation creates one or more data streams.

Configuration options

delete-data-stream

The delete-data-stream operation deletes one or more data streams.

Configuration options

create-composable-template

The create-composable-template operation creates one or more composable index templates.

Configuration options

delete-composable-template

The delete-composable-template operation deletes one or more composable index templates and, optionally, the indexes they match.

Configuration options

create-component-template

The create-component-template operation creates one or more component templates.

Configuration options

delete-component-template

The delete-component-template operation deletes one or more component templates.

Configuration options

create-index-template

The create-index-template operation creates one or more legacy index templates.

Configuration options

delete-index-template

The delete-index-template operation deletes one or more legacy index templates and, optionally, the indexes they match.

Configuration options

shrink-index

The shrink-index operation uses the Shrink Index API to reduce the number of primary shards in an index.

Usage

{
  "name": "shrink-index",
  "operation-type": "shrink-index",
  "source-index": "my-index",
  "target-index": "my-index-shrunk",
  "target-body": {
    "settings": {
      "index.number_of_shards": 1,
      "index.codec": "best_compression"
    }
  }
}

Configuration options

create-snapshot-repository

The create-snapshot-repository operation creates a snapshot repository.

Configuration options

delete-snapshot-repository

The delete-snapshot-repository operation deletes a snapshot repository.

Configuration options

create-snapshot

The create-snapshot operation creates a snapshot.

Configuration options

wait-for-snapshot-create

The wait-for-snapshot-create operation polls snapshot status until the snapshot reaches SUCCESS. If the snapshot ends in FAILED, the operation raises an assertion error. The reported metrics include snapshot size in bytes, throughput, and duration.

Configuration options

restore-snapshot

The restore-snapshot operation restores a snapshot.

Configuration options

wait-for-recovery

The wait-for-recovery operation waits until index recovery completes by polling the Index Recovery API.

Configuration options

submit-async-search

The submit-async-search operation submits an asynchronous search request and stores the returned asynchronous search ID under the operation name so later operations in the same composite can fetch or delete it.

Configuration options

get-async-search

The get-async-search operation retrieves results from one or more previously submitted asynchronous searches. The operation succeeds only when every referenced search has completed (is_running: false).

Configuration options

delete-async-search

The delete-async-search operation deletes one or more asynchronous search results.

Configuration options

create-point-in-time

The create-point-in-time operation creates a Point in Time (PIT) for an index. The resulting PIT ID is stored under the operation name, so later operations in the same composite can reference it.

Configuration options

delete-point-in-time

The delete-point-in-time operation deletes one or more PITs. If with-point-in-time-from is set, only the PIT created by the named operation is deleted; otherwise, every PIT on the cluster is deleted.

Configuration options

list-all-point-in-time

The list-all-point-in-time operation lists all active PITs on the cluster.

Configuration options

create-transform

The create-transform operation creates an index transform.

Configuration options

start-transform

The start-transform operation starts a previously created transform.

Configuration options

wait-for-transform

The wait-for-transform operation polls the transform status until the transform reaches its next checkpoint (or stops, depending on options).

Configuration options

delete-transform

The delete-transform operation deletes a previously created transform. Missing transforms (HTTP 404) are ignored so this operation is safe to use as a cleanup step.

Configuration options

train-knn-model

The train-knn-model operation trains a k-NN model using the Train Model API and polls until training completes.

Configuration options

delete-knn-model

The delete-knn-model operation deletes a trained k-NN model.

Configuration options

register-ml-model

The register-ml-model operation registers a machine learning model using the ML Commons API. The runner first searches for a model with a matching name; if one already exists, the existing model ID is reused. Otherwise it submits a new registration request and polls until it completes. The resulting model ID is written to model_id.json in the working directory so later operations can consume it.

You can supply the request body in one of two ways: by setting model-config-file to a path on disk, or by setting model-name, model-version, and model-format and letting OpenSearch Benchmark assemble the body for you.

Configuration options

deploy-ml-model

The deploy-ml-model operation deploys (loads) a previously registered ML model. The model ID is read from model_id.json in the working directory (typically written by an earlier register-ml-model operation), so this operation takes no required parameters.

Configuration options

create-ml-connector

The create-ml-connector operation creates an ML connector for remote model integration.

Configuration options

delete-ml-connector

The delete-ml-connector operation looks up an ML connector by name and deletes it. If no connector matches the supplied name, the operation is a no-op.

Configuration options

register-remote-ml-model

The register-remote-ml-model operation registers a remote ML model backed by an ML connector. It polls the registration task until it completes, then writes the resulting model ID to model_id.json in the working directory so later operations can consume it.

If connector_id is not set in the request body, OpenSearch Benchmark reads it from connector_id.json in the working directory (typically written by an earlier create-ml-connector operation).

Configuration options

delete-ml-model

The delete-ml-model operation looks up models by name, undeploys any that are currently deployed, and then deletes them. Multiple models with the same name (for example, across versions) are all deleted.

Configuration options

raw-request

The raw-request operation sends an arbitrary HTTP request to OpenSearch. Use this for operations not covered by a dedicated operation type.

Usage

{
  "name": "check-shard-count",
  "operation-type": "raw-request",
  "method": "GET",
  "path": "/_cat/shards?v",
  "body": {}
}

Configuration options

sleep

The sleep operation pauses execution for a specified duration.

Usage

{
  "name": "wait-before-search",
  "operation-type": "sleep",
  "duration": 30
}

Configuration options

composite

The composite operation executes a structured group of inner operations as a single measured unit, optionally with parallel sub-streams. This is useful when several requests need to share state (for example, creating a PIT, searching with it, and deleting it) or when you want to measure end-to-end timings across a small workflow.

Only a subset of operation types can appear inside composite: create-point-in-time, delete-point-in-time, list-all-point-in-time, search, paginated-search, raw-request, sleep, submit-async-search, get-async-search, and delete-async-search. Nested stream blocks run in parallel.

Usage

{
  "name": "pit-search-workflow",
  "operation-type": "composite",
  "requests": [
    { "operation-type": "create-point-in-time", "name": "pit", "index": "my-index" },
    { "operation-type": "search", "body": { "query": { "match_all": {} } } },
    { "operation-type": "delete-point-in-time", "with-point-in-time-from": "pit" }
  ]
}

Configuration options

produce-stream-message

The produce-stream-message operation publishes messages to a configured message-producer (for example, a Kafka producer) for streaming ingestion benchmarks. The body is split on newlines, and each non-metadata line is sent as an individual message. Producers are configured at the workload or cluster level — the runner itself takes only the producer reference and the message payload.

Configuration options

proto-bulk

The proto-bulk operation sends bulk index requests using the gRPC transport instead of HTTP REST. Requires gRPC to be enabled on the target cluster.

Configuration options

Same as bulk, but uses gRPC serialization (Protocol Buffers) instead of JSON over HTTP. Use --grpc-target-hosts to specify the gRPC endpoint.

proto-search

The proto-search operation sends search requests using gRPC transport.

Configuration options

Same as search, but uses gRPC. Use --grpc-target-hosts to specify the gRPC endpoint.

proto-vector-search

The proto-vector-search operation sends vector search requests using gRPC transport.

Configuration options

Same as vector-search, but uses gRPC. Use --grpc-target-hosts to specify the gRPC endpoint.

proto-bulk-vector-data-set

The proto-bulk-vector-data-set operation bulk-indexes vector data sets using the gRPC transport. It is the gRPC counterpart to bulk-vector-data-set.

Configuration options

Same as bulk-vector-data-set, but uses gRPC serialization (Protocol Buffers) instead of JSON over HTTP. Use --grpc-target-hosts to specify the gRPC endpoint.