Link Search Menu Expand Document Documentation Menu
Documentation

ISM API

Use the index state management operations to programmatically work with policies and managed indexes.

Table of contents

Create policy

Introduced 1.0

Creates a policy.

Example request

PUT _plugins/_ism/policies/policy_1
{
  "policy": {
    "description": "ingesting logs",
    "default_state": "ingest",
    "states": [
      {
        "name": "ingest",
        "actions": [
          {
            "rollover": {
              "min_doc_count": 5
            }
          }
        ],
        "transitions": [
          {
            "state_name": "search"
          }
        ]
      },
      {
        "name": "search",
        "actions": [],
        "transitions": [
          {
            "state_name": "delete",
            "conditions": {
              "min_index_age": "5m"
            }
          }
        ]
      },
      {
        "name": "delete",
        "actions": [
          {
            "delete": {}
          }
        ],
        "transitions": []
      }
    ]
  }
}

Example response

{
  "_id": "policy_1",
  "_version": 1,
  "_primary_term": 1,
  "_seq_no": 7,
  "policy": {
    "policy": {
      "policy_id": "policy_1",
      "description": "ingesting logs",
      "last_updated_time": 1577990761311,
      "schema_version": 1,
      "error_notification": null,
      "default_state": "ingest",
      "states": [
        {
          "name": "ingest",
          "actions": [
            {
              "rollover": {
                "min_doc_count": 5
              }
            }
          ],
          "transitions": [
            {
              "state_name": "search"
            }
          ]
        },
        {
          "name": "search",
          "actions": [],
          "transitions": [
            {
              "state_name": "delete",
              "conditions": {
                "min_index_age": "5m"
              }
            }
          ]
        },
        {
          "name": "delete",
          "actions": [
            {
              "delete": {}
            }
          ],
          "transitions": []
        }
      ]
    }
  }
}

Add policy

Introduced 1.0

Adds a policy to an index. This operation does not change the policy if the index already has one.

Example request

POST _plugins/_ism/add/index_1
{
  "policy_id": "policy_1"
}

Example response

{
  "updated_indices": 1,
  "failures": false,
  "failed_indices": []
}

If you use a wildcard * while adding a policy to an index, the ISM plugin interprets * as all indexes, including system indexes like .opendistro-security, which stores users, roles, and tenants. A delete action in your policy might accidentally delete all user roles and tenants in your cluster. Don’t use the broad * wildcard, and instead add a prefix, such as my-logs*, when specifying indexes with the _ism/add API.

Update policy

Introduced 1.0

Updates a policy. Use the seq_no and primary_term parameters to update an existing policy. If these numbers don’t match the existing policy or the policy doesn’t exist, ISM throws an error.

It’s possible that the policy currently applied to your index isn’t the most up-to-date policy available. To see what policy is currently applied to your index, see Explain index. To get the most up-to-date version of a policy, see Get policy.

Example request

PUT _plugins/_ism/policies/policy_1?if_seq_no=7&if_primary_term=1
{
  "policy": {
    "description": "ingesting logs",
    "default_state": "ingest",
    "states": [
      {
        "name": "ingest",
        "actions": [
          {
            "rollover": {
              "min_doc_count": 5
            }
          }
        ],
        "transitions": [
          {
            "state_name": "search"
          }
        ]
      },
      {
        "name": "search",
        "actions": [],
        "transitions": [
          {
            "state_name": "delete",
            "conditions": {
              "min_index_age": "5m"
            }
          }
        ]
      },
      {
        "name": "delete",
        "actions": [
          {
            "delete": {}
          }
        ],
        "transitions": []
      }
    ]
  }
}

Example response

{
  "_id": "policy_1",
  "_version": 2,
  "_primary_term": 1,
  "_seq_no": 10,
  "policy": {
    "policy": {
      "policy_id": "policy_1",
      "description": "ingesting logs",
      "last_updated_time": 1577990934044,
      "schema_version": 1,
      "error_notification": null,
      "default_state": "ingest",
      "states": [
        {
          "name": "ingest",
          "actions": [
            {
              "rollover": {
                "min_doc_count": 5
              }
            }
          ],
          "transitions": [
            {
              "state_name": "search"
            }
          ]
        },
        {
          "name": "search",
          "actions": [],
          "transitions": [
            {
              "state_name": "delete",
              "conditions": {
                "min_index_age": "5m"
              }
            }
          ]
        },
        {
          "name": "delete",
          "actions": [
            {
              "delete": {}
            }
          ],
          "transitions": []
        }
      ]
    }
  }
}

Get policy

Introduced 1.0

Gets the policy by policy_id.

Example request

GET _plugins/_ism/policies/policy_1

Example response

{
  "_id": "policy_1",
  "_version": 2,
  "_seq_no": 10,
  "_primary_term": 1,
  "policy": {
    "policy_id": "policy_1",
    "description": "ingesting logs",
    "last_updated_time": 1577990934044,
    "schema_version": 1,
    "error_notification": null,
    "default_state": "ingest",
    "states": [
      {
        "name": "ingest",
        "actions": [
          {
            "rollover": {
              "min_doc_count": 5
            }
          }
        ],
        "transitions": [
          {
            "state_name": "search"
          }
        ]
      },
      {
        "name": "search",
        "actions": [],
        "transitions": [
          {
            "state_name": "delete",
            "conditions": {
              "min_index_age": "5m"
            }
          }
        ]
      },
      {
        "name": "delete",
        "actions": [
          {
            "delete": {}
          }
        ],
        "transitions": []
      }
    ]
  }
}

Get policies

Introduced 1.0

Gets a list of policies. This API accepts search parameters to filter and paginate the results.

Query parameters

The following table lists the available query parameters. All query parameters are optional.

Parameter Data type Description
size Integer The number of policies to return.
from Integer The starting position for pagination.
sortField String The field by which to sort the results.
sortOrder String The sort order for the results. Valid values are asc (ascending) and desc (descending).
queryString String A query string used to filter policies by name or other attributes. See Query string query.

Example request

GET _plugins/_ism/policies

Example response

{
  "policies": [
    {
      "_id": "policy_1",
      "_version": 2,
      "_seq_no": 10,
      "_primary_term": 1,
      "policy": {
        "policy_id": "policy_1",
        "description": "ingesting logs",
        "last_updated_time": 1577990934044,
        "schema_version": 1,
        "error_notification": null,
        "default_state": "ingest",
        "states": [
          {
            "name": "ingest",
            "actions": [
              {
                "rollover": {
                  "min_doc_count": 5
                }
              }
            ],
            "transitions": [
              {
                "state_name": "search"
              }
            ]
          },
          {
            "name": "search",
            "actions": [],
            "transitions": [
              {
                "state_name": "delete",
                "conditions": {
                  "min_index_age": "5m"
                }
              }
            ]
          },
          {
            "name": "delete",
            "actions": [
              {
                "delete": {}
              }
            ],
            "transitions": []
          }
        ]
      }
    },
    {
      "_id": "policy_2",
      "_version": 3,
      "_seq_no": 11,
      "_primary_term": 1,
      "policy": {
        "policy_id": "policy_2",
        "description": "ingesting logs",
        "last_updated_time": 1577990934042,
        "schema_version": 1,
        "error_notification": null,
        "default_state": "ingest",
        "states": [
          {
            "name": "ingest",
            "actions": [
              {
                "rollover": {
                  "min_doc_count": 5
                }
              }
            ],
            "transitions": [
              {
                "state_name": "search"
              }
            ]
          },
          {
            "name": "search",
            "actions": [],
            "transitions": [
              {
                "state_name": "delete",
                "conditions": {
                  "min_index_age": "5m"
                }
              }
            ]
          },
          {
            "name": "delete",
            "actions": [
              {
                "delete": {}
              }
            ],
            "transitions": []
          }
        ]
      }
    }
  ]
}

Remove policy from index

Introduced 1.0

Removes any ISM policy from the index.

Example request

POST _plugins/_ism/remove/index_1

Example response

{
  "updated_indices": 1,
  "failures": false,
  "failed_indices": []
}

Update managed index policy

Introduced 1.0

Updates the managed index policy to a new policy (or to a new version of the policy). You can use an index pattern to update multiple indexes at once. When updating multiple indexes, you might want to include a state filter to only affect certain managed indexes. The change policy filters out all the existing managed indexes and only applies the change to the ones in the state that you specify. You can also explicitly specify the state that the managed index transitions to after the change policy takes effect.

A policy change is an asynchronous background process. The changes are queued and are not executed immediately by the background process. This delay in execution protects the currently running managed indexes from being put into a broken state. If the policy you are changing to has only some small configuration changes, then the change takes place immediately. For example, if the policy changes the min_index_age parameter in a rollover condition from 1000d to 100d, this change takes place immediately in its next execution. If the change modifies the state, actions, or the order of actions of the current state the index is in, then the change happens at the end of its current state before transitioning to a new state.

In this example, the policy applied on the index_1 index is changed to policy_1, which could either be a completely new policy or an updated version of its existing policy. The process only applies the change if the index is currently in the searches state. After this change in policy takes place, index_1 transitions to the delete state.

Example request

POST _plugins/_ism/change_policy/index_1
{
  "policy_id": "policy_1",
  "state": "delete",
  "include": [
    {
      "state": "searches"
    }
  ]
}

Example response

{
  "updated_indices": 0,
  "failures": false,
  "failed_indices": []
}

Retry failed index

Introduced 1.0

Retries the failed action for an index. For the retry call to succeed, ISM must manage the index, and the index must be in a failed state. You can use index patterns (*) to retry multiple failed indexes.

Example request

POST _plugins/_ism/retry/index_1
{
  "state": "delete"
}

Example response

{
  "updated_indices": 0,
  "failures": false,
  "failed_indices": []
}

Explain index

Introduced 1.0

Gets the current state of the index. You can use index patterns to get the status of multiple indexes.

Example request

GET _plugins/_ism/explain/index_1

Example response

{
  "index_1": {
    "index.plugins.index_state_management.policy_id": "policy_1"
  }
}

Optionally, you can add the show_policy parameter to your request’s path to get the policy that is currently applied to your index, which is useful for seeing whether the policy applied to your index is the latest one. To get the most up-to-date policy, see Get Policy API.

Example request

GET _plugins/_ism/explain/index_1?show_policy=true

Example response

{
  "index_1": {
    "index.plugins.index_state_management.policy_id": "sample-policy",
    "index.opendistro.index_state_management.policy_id": "sample-policy",
    "index": "index_1",
    "index_uuid": "gCFlS_zcTdih8xyxf3jQ-A",
    "policy_id": "sample-policy",
    "enabled": true,
    "policy": {
      "policy_id": "sample-policy",
      "description": "ingesting logs",
      "last_updated_time": 1647284980148,
      "schema_version": 13,
      "error_notification": null,
      "default_state": "ingest",
      "states": [...],
      "ism_template": null
    }
  },
  "total_managed_indices": 1
}

The plugins.index_state_management.policy_id setting is deprecated starting from ODFE version 1.13.0. We retain this field in the response API for consistency.

Explain index with filtering

Introduced 2.12

You can use the POST method with the Explain API to filter the results based on specific criteria. This allows you to query indexes based on their policy ID, current state, or action type.

Endpoints

POST _plugins/_ism/explain/{index}

Request body

The request body supports the following optional filters. If a filter is not specified, indexes with any value for that parameter are included in the results. The API returns only indexes that match all specified filters.

Parameter Type Description
policy_id String Filter results to show only indexes managed by the specified policy ID.
state String Filter results to show only indexes currently in the specified state.
action_type String Filter results to show only indexes currently executing the specified action type.
failed Boolean Filter results to show only failed managed indexes.

Example request: Filter by policy ID

POST _plugins/_ism/explain/log-*
{
  "filter": {
    "policy_id": "hot-warm-delete-policy"
  }
}

Example request: Filter by state and action type

POST _plugins/_ism/explain/app-*
{
  "filter": {
    "state": "warm",
    "action_type": "allocation"
  }
}

Example request: Filter by all criteria

POST _plugins/_ism/explain/data-*
{
  "filter": {
    "policy_id": "data-lifecycle-policy",
    "state": "hot",
    "action_type": "rollover"
  }
}

Example response

Response 
{
  "test-logs-001": {
    "index.plugins.index_state_management.policy_id": "test-lifecycle-policy",
    "index": "test-logs-001",
    "index_uuid": "LmJgKNatQZWHQu-qIHlcJw",
    "policy_id": "test-lifecycle-policy",
    "enabled": true,
    "policy": {
      "policy_id": "test-lifecycle-policy",
      "description": "Lifecycle policy for log data: hot -> warm -> cold -> delete",
      "last_updated_time": 1730308440926,
      "schema_version": 18,
      "error_notification": null,
      "default_state": "hot",
      "states": [
        {
          "name": "hot",
          "actions": [
            {
              "rollover": {
                "min_doc_count": 10000,
                "min_size": "1gb",
                "min_index_age": "1d"
              }
            }
          ],
          "transitions": [
            {
              "state_name": "warm",
              "conditions": {
                "min_index_age": "7d"
              }
            }
          ]
        },
        {
          "name": "warm",
          "actions": [
            {
              "replica_count": {
                "number_of_replicas": 0
              }
            }
          ],
          "transitions": [
            {
              "state_name": "cold",
              "conditions": {
                "min_index_age": "30d"
              }
            }
          ]
        },
        {
          "name": "cold",
          "actions": [],
          "transitions": [
            {
              "state_name": "delete",
              "conditions": {
                "min_index_age": "90d"
              }
            }
          ]
        },
        {
          "name": "delete",
          "actions": [
            {
              "delete": {}
            }
          ],
          "transitions": []
        }
      ],
      "ism_template": null
    },
    "policy_seq_no": 0,
    "policy_primary_term": 1,
    "rolled_over": false,
    "index_creation_date": 1730308447399,
    "state": {
      "name": "hot",
      "start_time": 1730308447644
    },
    "action": {
      "name": "rollover",
      "start_time": 1730308447644,
      "index": 0,
      "failed": false,
      "consumed_retries": 0,
      "last_retry_time": 0
    },
    "step": {
      "name": "attempt_rollover",
      "start_time": 1730308447644,
      "step_status": "starting"
    },
    "retry_info": {
      "failed": false,
      "consumed_retries": 0
    },
    "info": {
      "message": "Currently checking rollover conditions"
    },
    "enabled": true,
    "enabled_time": 1730308447644
  },
  "total_managed_indices": 1
}

Simulate policy

Introduced 3.7

Previews how a policy would apply to one or more indexes without making any changes. For each index, the response contains the current state, the next action to execute, the evaluation of every transition condition, and which state the index would move to next. Use this endpoint to validate a policy before attaching it or to debug why an index is not transitioning as expected.

Exactly one of policy_id or policy must be provided.

Endpoints

POST _plugins/_ism/simulate

Request body fields

The following table lists the available request body fields.

Field Type Required Description
policy_id String Conditional The ID of a stored ISM policy to simulate. Required when policy is not provided.
policy Object Conditional An inline policy definition to simulate without saving it. Required when policy_id is not provided.
indices Array of strings Yes The index names or wildcard patterns to simulate against. Wildcard patterns are expanded to matching concrete indexes. Patterns that match no indexes are silently ignored. Concrete index names that do not exist return an error in the per-index result.

Example request: Simulate a stored policy

POST _plugins/_ism/simulate
{
  "policy_id": "my-lifecycle-policy",
  "indices": ["logs-2024-01-*", "logs-2024-02-01"]
}

Example request: Simulate an inline policy

POST _plugins/_ism/simulate
{
  "policy": {
    "description": "hot-warm lifecycle",
    "default_state": "hot",
    "states": [
      {
        "name": "hot",
        "actions": [],
        "transitions": [
          {
            "state_name": "warm",
            "conditions": { "min_index_age": "7d" }
          }
        ]
      },
      { "name": "warm", "actions": [], "transitions": [] }
    ]
  },
  "indices": ["my-index"]
}

Response body fields

The following table lists all response body fields. The response contains a simulate_results array with one entry per index.

Field Type Description
index_name String The name of the index.
index_uuid String The UUID of the index. null when the index was not found.
policy_id String The ID of the policy used for simulation. Empty string for inline policies.
is_managed Boolean Whether the index is managed by ISM.
current_state String The state the index is in, or would start in for unmanaged indexes. Omitted when error is present.
current_action String The action that would execute next in the current state. Omitted when error is present.
transition_evaluation Array Per-transition condition evaluations. Present only when the index is in the transition phase (no pending actions). Omitted when error is present.
next_state String The state the index would transition to (first met condition). null if no condition is met. Omitted when error is present.
error String An index-level error message. Present when the index does not exist or another index-level error occurs. When this field is present, all other fields except index_name, index_uuid, policy_id, and is_managed are omitted.

Each object in transition_evaluation contains the following fields.

Field Type Description
state_name String The target state this transition would move the index to.
condition_met Boolean Whether the transition condition is satisfied.
condition_type String The condition type (for example, min_index_age, min_doc_count, min_size). Set to unconditional for transitions with no conditions.
current_value String The current value of the metric being checked, formatted as a readable string (for example, "3d 4h"). Omitted for unconditional transitions.
required_value String The threshold required by the condition, formatted as a readable string (for example, "7d"). Omitted for unconditional transitions.

Example response

{
  "simulate_results": [
    {
      "index_name": "logs-2024-01-15",
      "index_uuid": "gCFlS_zcTdih8xyxf3jQ-A",
      "policy_id": "my-lifecycle-policy",
      "is_managed": false,
      "current_state": "hot",
      "current_action": "transition",
      "transition_evaluation": [
        {
          "state_name": "warm",
          "condition_met": true,
          "condition_type": "min_index_age",
          "current_value": "10d",
          "required_value": "7d"
        }
      ],
      "next_state": "warm"
    },
    {
      "index_name": "logs-2024-02-01",
      "index_uuid": "LmJgKNatQZWHQu-qIHlcJw",
      "policy_id": "my-lifecycle-policy",
      "is_managed": false,
      "current_state": "hot",
      "current_action": "transition",
      "transition_evaluation": [
        {
          "state_name": "warm",
          "condition_met": false,
          "condition_type": "min_index_age",
          "current_value": "3d 2h",
          "required_value": "7d"
        }
      ],
      "next_state": null
    }
  ]
}

Example response: Index not found

When a concrete index name does not exist in the cluster, the result contains an error field instead of state information.

{
  "simulate_results": [
    {
      "index_name": "nonexistent-index",
      "index_uuid": null,
      "policy_id": "my-lifecycle-policy",
      "is_managed": false,
      "error": "Index 'nonexistent-index' not found in cluster"
    }
  ]
}

Delete policy

Introduced 1.0

Deletes the policy by policy_id.

Example request

DELETE _plugins/_ism/policies/policy_1

Example response

{
  "_index": ".opendistro-ism-config",
  "_id": "policy_1",
  "_version": 3,
  "result": "deleted",
  "forced_refresh": true,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "_seq_no": 15,
  "_primary_term": 1
}

Error prevention validation

Introduced 2.4

ISM runs actions automatically. However, actions can fail for various reasons. You can use error prevention validation to test an action and identify potential failures.

To enable error prevention validation, set plugins.index_state_management.action_validation.enabled to true:

PUT _cluster/settings
{
   "persistent": {
      "plugins.index_state_management.action_validation.enabled": true
   }
}

Example response

The following response confirms the setting was updated:

{
  "acknowledged" : true,
  "persistent" : {
    "plugins" : {
      "index_state_management" : {
        "validation_action" : {
          "enabled" : "true"
        }
      }
    }
  },
  "transient" : { }
}

To retrieve the error prevention validation status and message, pass validate_action=true to the _plugins/_ism/explain endpoint:

GET _plugins/_ism/explain/test-000001?validate_action=true

Example response

The response contains an additional validate object with a validation message and status:

{
  "test-000001" : {
    "index.plugins.index_state_management.policy_id" : "test_rollover",
    "index.opendistro.index_state_management.policy_id" : "test_rollover",
    "index" : "test-000001",
    "index_uuid" : "CgKsxFmQSIa8dWqpbSJmyA",
    "policy_id" : "test_rollover",
    "policy_seq_no" : -2,
    "policy_primary_term" : 0,
    "rolled_over" : false,
    "index_creation_date" : 1667410460649,
    "state" : {
      "name" : "rollover",
      "start_time" : 1667410766045
    },
    "action" : {
      "name" : "rollover",
      "start_time" : 1667411127803,
      "index" : 0,
      "failed" : false,
      "consumed_retries" : 0,
      "last_retry_time" : 0
    },
    "step" : {
      "name" : "attempt_rollover",
      "start_time" : 1667411127803,
      "step_status" : "starting"
    },
    "retry_info" : {
      "failed" : true,
      "consumed_retries" : 0
    },
    "info" : {
      "message" : "Previous action was not able to update IndexMetaData."
    },
    "enabled" : false,
    "validate" : {
      "validation_message" : "Missing rollover_alias index setting [index=test-000001]",
      "validation_status" : "re_validating"
    }
  },
  "total_managed_indices" : 1
}

If you pass validate_action=false or omit the validate_action parameter, the response does not contain the validation status and message:

GET _plugins/_ism/explain/test-000001?validate_action=false


GET _plugins/_ism/explain/test-000001

Example response

{
  "test-000001" : {
    "index.plugins.index_state_management.policy_id" : "test_rollover",
    "index.opendistro.index_state_management.policy_id" : "test_rollover",
    "index" : "test-000001",
    "index_uuid" : "CgKsxFmQSIa8dWqpbSJmyA",
    "policy_id" : "test_rollover",
    "policy_seq_no" : -2,
    "policy_primary_term" : 0,
    "rolled_over" : false,
    "index_creation_date" : 1667410460649,
    "state" : {
      "name" : "rollover",
      "start_time" : 1667410766045
    },
    "action" : {
      "name" : "rollover",
      "start_time" : 1667411127803,
      "index" : 0,
      "failed" : false,
      "consumed_retries" : 0,
      "last_retry_time" : 0
    },
    "step" : {
      "name" : "attempt_rollover",
      "start_time" : 1667411127803,
      "step_status" : "starting"
    },
    "retry_info" : {
      "failed" : true,
      "consumed_retries" : 0
    },
    "info" : {
      "message" : "Previous action was not able to update IndexMetaData."
    },
    "enabled" : false
  },
  "total_managed_indices" : 1
}
350 characters left

Have a question? .

Want to contribute? or .