schedule

The schedule element contains a list of tasks that are run in a specified order during the benchmark test. Each task is an operation supported by OpenSearch Benchmark.

You can define schedule in either of the following locations:

• At the top level of workload.json. Use this form when the workload defines a single benchmarking scenario. OpenSearch Benchmark treats the schedule as an implicit default test procedure.
• Inside a test procedure, in the test_procedures element. Use this form when the workload defines multiple scenarios, each with its own name, description, and schedule.

Example

The following schedule creates an index, waits for the cluster to become healthy, indexes documents in bulk, and then runs a match_all query:

  "schedule": [
    {
      "operation": {
        "operation-type": "create-index"
      }
    },
    {
      "operation": {
        "operation-type": "cluster-health",
        "request-params": {
          "wait_for_status": "green"
        },
        "retry-until-success": true
      }
    },
    {
      "operation": {
        "operation-type": "bulk",
        "bulk-size": 5000
      },
      "warmup-time-period": 120,
      "clients": 8
    },
    {
      "operation": {
        "name": "query-match-all",
        "operation-type": "search",
        "body": {
          "query": {
            "match_all": {}
          }
        }
      },
      "iterations": 1000,
      "target-throughput": 100
    }
  ]

According to this schedule, the actions run in the following order:

1. The create-index operation creates an index. The index remains empty until the bulk operation adds documents with benchmarked data.
2. The cluster-health operation assesses the cluster’s health before running the workload. In this example, the workload waits until the cluster’s health status is green.
   • The bulk operation runs the bulk API to index 5000 documents simultaneously.
   • Before benchmarking, the workload waits until the specified warmup-time-period passes. In this example, the warmup period is 120 seconds.
3. The clients field defines the number of clients, in this example, eight, that run the bulk indexing operation concurrently.
4. The search operation runs a match_all query to match all documents after they have been indexed by the bulk API using the specified clients.
   • The iterations field defines the number of times each client runs the search operation. The benchmark report automatically adjusts the percentile numbers based on this number. To generate a precise percentile, the benchmark needs to run at least 1,000 iterations.
   • The target-throughput field defines the number of requests per second performed by each client. This setting can help reduce benchmark latency. For example, a target-throughput of 100 requests divided by 8 clients means that each client issues 12 requests per second. For more information about how target throughput is defined in OpenSearch Benchmark, see Target throughput.

Defining tasks

The schedule element defines tasks using the methods described in this section.

Using the operations element

The following example defines a force-merge and match-all query task using the operations element. The force-merge operation does not use any parameters, so only the name and operation-type are needed. The match-all-query parameter requires a query body and operation-type.

Operations defined in the operations element can be reused in the schedule more than once:

{
  "operations": [
    {
      "name": "force-merge",
      "operation-type": "force-merge"
    },
    {
      "name": "match-all-query",
      "operation-type": "search",
      "body": {
        "query": {
          "match_all": {}
        }
      }
    }
  ],
  "schedule": [
    {
      "operation": "force-merge",
      "clients": 1
    },
    {
      "operation": "match-all-query",
      "clients": 4,
      "warmup-iterations": 1000,
      "iterations": 1000,
      "target-throughput": 100
    }
  ]
}

For the full list of available operation types, see operations.

Defining operations inline

If you don’t want to reuse an operation in the schedule, you can define operations inside the schedule element, as shown in the following example:

{
  "schedule": [
    {
      "operation": {
        "name": "force-merge",
        "operation-type": "force-merge"
      },
      "clients": 1
    },
    {
      "operation": {
        "name": "match-all-query",
        "operation-type": "search",
        "body": {
          "query": {
            "match_all": {}
          }
        }
      },
      "clients": 4,
      "warmup-iterations": 1000,
      "iterations": 1000,
      "target-throughput": 100
    }
  ]
}

Task options

Each task contains the following options.

Target options

OpenSearch Benchmark requires one of the following options when running a task.

Iteration-based options

Iteration-based options determine the number of times that an operation should run. They can also define the number of iterative runs when tasks are run in parallel. To configure an iteration-based schedule, use the following options.

Time-based options

Time-based options determine the duration of time, in seconds, for which operations should run. This is ideal for batch-style operations, which may require an additional warmup period.

To configure a time-based schedule, use the following options.

Parallel tasks

The parallel element concurrently runs tasks wrapped inside the element.

When running tasks in parallel, each task requires the client option in order to ensure that clients inside your benchmark are reserved for that task. Otherwise, when the client option is specified inside the parallel element without a connection to the task, the benchmark uses that number of clients for all tasks.

In the following example, parallel-task-1 and parallel-task-2 execute a bulk operation concurrently:

{
  "name": "parallel-any",
  "description": "Workload completed-by property",
  "schedule": [
    {
      "parallel": {
        "tasks": [
          {
            "name": "parellel-task-1",
            "operation": {
              "operation-type": "bulk",
              "bulk-size": 1000
            },
            "clients": 8
          },
          {
            "name": "parellel-task-2",
            "operation": {
              "operation-type": "bulk",
              "bulk-size": 500
            },
            "clients": 8
          }
        ]
      }
    }
  ]
}

The parallel element supports all schedule parameters, in addition to the following options.