Explain API

Introduced 1.0

The Explain API returns detailed information about why a specific document matches or does not match a query. This API helps you understand how OpenSearch calculates the relevance score (_score) for each search result, making it an essential tool for debugging search relevance issues and optimizing queries.

OpenSearch uses a probabilistic ranking framework called Okapi BM25 to calculate relevance scores. Okapi BM25 is based on the original term frequency/inverse document frequency (TF/IDF) framework used by Apache Lucene.

Using the Explain API is expensive in terms of both resources and time. For production clusters, we recommend using it sparingly for the purpose of troubleshooting.

Limitations

The Explain API does not support the search_pipeline parameter. Sending a request with an inline or named search pipeline results in a parsing_exception error. This applies to all query types.

To use search pipeline processors with explain, provide the explain=true query parameter to the Search API. For more information about using explain with hybrid queries, see Hybrid search explain.

Endpoints

GET  /{index}/_explain/{id}
POST /{index}/_explain/{id}

Path parameters

The following table lists the available path parameters.

Query parameters

You must specify the index and document ID. All other parameters are optional.

Request body fields

The request body contains the query to explain against the specified document. The following table lists the available request body fields.

Example: Explaining a match query

The following example explains why document 1 in the products index matches a query for the term “computer” in the name field:

POST /products/_explain/1
{
  "query": {
    "match": {
      "name": "computer"
    }
  }
}

response = client.explain(
  id = "1",
  index = "products",
  body =   {
    "query": {
      "match": {
        "name": "computer"
      }
    }
  }
)

Example response

The Explain API returns a response that includes whether the document matched the query and a detailed breakdown of the score calculation. The explanation includes the BM25 scoring formula components: term frequency (tf), inverse document frequency (idf), and field length normalization:

{
  "_index": "products",
  "_id": "1",
  "matched": true,
  "explanation": {
    "value": 0.31506687,
    "description": "weight(name:computer in 0) [PerFieldSimilarity], result of:",
    "details": [
      {
        "value": 0.31506687,
        "description": "score(freq=1.0), computed as boost * idf * tf from:",
        "details": [
          {
            "value": 0.6931472,
            "description": "idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from:",
            "details": [
              {
                "value": 2,
                "description": "n, number of documents containing term",
                "details": []
              },
              {
                "value": 4,
                "description": "N, total number of documents with field",
                "details": []
              }
            ]
          },
          {
            "value": 0.45454544,
            "description": "tf, computed as freq / (freq + k1 * (1 - b + b * dl / avgdl)) from:",
            "details": [
              {
                "value": 1.0,
                "description": "freq, occurrences of term within document",
                "details": []
              },
              {
                "value": 1.2,
                "description": "k1, term saturation parameter",
                "details": []
              },
              {
                "value": 0.75,
                "description": "b, length normalization parameter",
                "details": []
              },
              {
                "value": 2.0,
                "description": "dl, length of field",
                "details": []
              },
              {
                "value": 2.0,
                "description": "avgdl, average length of field",
                "details": []
              }
            ]
          }
        ]
      }
    ]
  }
}

Example: Using the query string parameter

You can use the q parameter to specify a query using Lucene query string syntax instead of providing a request body. The following example explains why document 1 matches the query string “name:laptop”:

GET /products/_explain/1?q=name:laptop

response = client.explain(
  id = "1",
  index = "products",
  params = { "q": "name:laptop" },
  body = { "Insert body here" }
)

Response body fields

The response contains detailed information about whether the document matched the query and how the relevance score was calculated. The following table lists the response body fields.

BM25 scoring components

The explanation details include the following BM25 scoring components.

The final relevance score is calculated by multiplying these components together: score = boost * idf * tf. The values are calculated and stored at index time when a document is added or updated and may have small inaccuracies based on shard-level statistics.