Search API (gRPC)

Introduced 3.0

This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated GitHub issue.

The gRPC Search API provides a performant, binary interface for running queries using protocol buffers over gRPC. It mirrors the capabilities of the HTTP Search API while benefiting from protobuf-typed contracts and gRPC transport. The gRPC APIs are ideal for low-latency, high-throughput applications.

Prerequisite

To submit gRPC requests, you must have a set of protobufs on the client side. For ways to obtain the protobufs, see Using gRPC APIs.

gRPC service and method

gRPC Document APIs reside in the SearchService.

You can submit search requests by invoking the Search gRPC method within the SearchService. The method takes a SearchRequest and returns a SearchResponse.

See Supported Queries for currently supported search queries. Additional query types will be supported in future versions.

Request fields

The gRPC Search API supports the following request fields.

SearchRequest fields

The SearchRequest message accepts the following fields. All fields are optional.

SearchRequestBody fields

The SearchRequestBody message accepts the following fields. All fields are optional.

DerivedField fields

The DerivedField message is used for dynamically computed fields that are calculated during search execution. It accepts the following fields.

QueryContainer fields

QueryContainer is the entry point for all supported query types.

Exactly one of the following fields must be provided in each QueryContainer message.

Note that some query types are currently unsupported. See Supported queries for the current list of implemented query types.

Supported queries

The gRPC Search API supports the following query types:

• Term-level: exists, fuzzy, ids, prefix, range, regexp, term, terms, terms_set, wildcard
• Full-text: match, match_bool_prefix, match_phrase, match_phrase_prefix, multi_match
• Compound queries: bool, constant_score
• Geographic: geo_bounding_box, geo_distance
• Joining queries: nested
• Specialized queries: script

For more information about these query types, see Query DSL.

Term-level query fields

The following sections describe the fields for each term-level query message.

ExistsQuery fields

The ExistsQuery message accepts the following fields.

FuzzyQuery fields

The FuzzyQuery message accepts the following fields.

IdsQuery fields

The IdsQuery message accepts the following fields.

PrefixQuery fields

The PrefixQuery message accepts the following fields.

RangeQuery fields

The RangeQuery message is a oneof type that can contain either a NumberRangeQuery or a DateRangeQuery.

NumberRangeQuery fields

The NumberRangeQuery message accepts the following fields.

he NumberRangeQuery message accepts the following fields.

DateRangeQuery fields

The DateRangeQuery message accepts the following fields.

The DateRangeQuery message accepts the following fields.

RegexpQuery fields

The RegexpQuery message accepts the following fields.

TermQuery fields

The TermQuery message accepts the following fields.

TermsQuery fields

The TermsQuery message accepts the following fields.

TermsSetQuery fields

The TermsSetQuery message accepts the following fields.

WildcardQuery fields

The WildcardQuery message accepts the following fields.

Full-text query fields

The following sections describe the fields for each full-text query message.

The following sections describe the fields for each full-text query message.

MatchQuery fields

The MatchQuery message accepts the following fields.

MatchBoolPrefixQuery fields

The MatchBoolPrefixQuery message accepts the following fields.

MatchPhraseQuery fields

The MatchPhraseQuery message accepts the following fields.

MatchPhrasePrefixQuery fields

The MatchPhrasePrefixQuery message accepts the following fields.

MultiMatchQuery fields

The MultiMatchQuery message accepts the following fields.

MatchAllQuery fields

The MatchAllQuery message accepts the following fields.

MatchNoneQuery fields

The MatchNoneQuery message accepts the following fields.

Compound query fields

The following sections describe the fields for each compound query message.

The following sections describe the fields for each compound query message.

BoolQuery fields

The BoolQuery message accepts the following fields.

ConstantScoreQuery fields

The ConstantScoreQuery message accepts the following fields.

Joining query fields

The following sections describe the fields for each joining query message.

The following sections describe the fields for each joining query message.

NestedQuery fields

The NestedQuery message accepts the following fields.

Geographic query fields

The following sections describe the fields for each geographic query message.

The following sections describe the fields for each geographic query message.

GeoBoundingBoxQuery fields

A geo-bounding box query returns documents whose geopoints are within the bounding box specified in the query. The GeoBoundingBoxQuery message accepts the following fields.

GeoDistanceQuery fields

A geodistance query returns documents with geopoints that are within a specified distance from the provided geopoint. The GeoDistanceQuery message accepts the following fields.

Specialized query fields

The following sections describe the fields for each specialized query message.

The following sections describe the fields for each specialized query message.

ScriptQuery fields

A script query filters documents based on a custom condition written in the Painless scripting language. The ScriptQuery message accepts the following fields.

Common message values

The following sections describe the values used by common message types.

Fuzziness

The Fuzziness message is a oneof type that accepts the following values.

MultiTermQueryRewrite

The MultiTermQueryRewrite enum determines how OpenSearch rewrites and scores multi-term queries. It accepts the following values.

MinimumShouldMatch

The MinimumShouldMatch message is a oneof type that accepts the following values.

FieldValue

The FieldValue message represents a field value and accepts the following values.

InnerHits fields

The InnerHits message accepts the following fields.

All of the following examples show valid request payloads that can be sent to the SearchService/Search gRPC method.

Match all query

A match_all query returns all documents in the index. For example, the following request returns a maximum of 50 documents from the index:

{
  "search_request_body": {
    "query": {
      "match_all": {}
    },
    "size": 50
  }
}

Term query

A term query matches a single field with a specific term. For example, the following query searches for titles containing the word Rush:

{
  "index": "my_index",
  "search_request_body": {
    "query": {
      "term": {
        "field": "title",
        "value": {
          "string_value": "Rush"
        },
        "case_insensitive": true
      }
    }
  }
}

Terms query

A terms query matches documents in which a specific field contains any value from a list. For example, the following query searches for lines with the IDs 61809 and 61810:

{
 "search_request_body": {
   "query": {
     "terms": {
       "terms_lookup_field_string_array_map": {
         "line_id": {
           "string_array": {
             "string_array": ["61809", "61810"]
           }
         }
       }
     }
   }
 }
}

Terms query with a terms lookup

A terms query with a terms lookup is a specialized form of the terms query that allows you to fetch the terms for filtering from another document in your cluster rather than specifying them directly in the query. For example, the following request matches documents in the students index with every student whose id matches one of the values in the enrolled array:

{
  "search_request_body": {
    "query": {
      "terms": {
        "boost": 1.0,
        "terms_lookup_field_string_array_map": {
          "student_id": {
            "terms_lookup_field": {
              "index": "classes",
              "id": "101",
              "path": "enrolled"
            }
          }
        }
      }
    }
  }
}

Match none query

A match_none query matches none of the documents:

{
  "search_request_body": {
    "query": {
      "match_none": {}
    }
  }
}

Response fields

SearchResponse fields

The following table lists the supported fields for the SearchResponse message accepts the following fields.

The source documents are returned as bytes. Use Base64 decoding to read the _source field in the gRPC response.

HitsMetadata fields

The HitsMetadata object contains information about the search results, including the total number of matching documents and an array of individual document matches. It includes the following fields.

HitsMetadataHitsInner fields

Each HitsMetadataHitsInner represents a single document matched by the query and contains the following fields.

source is Base64 encoded and must be decoded to obtain the JSON document.

Example response

{
  "response_body": {
    "took": 64,
    "timed_out": false,
    "shards": {
      "successful": 1,
      "total": 1
    },
    "hits": {
      "total": {
        "total_hits": {
          "relation": "TOTAL_HITS_RELATION_EQ",
          "value": 1
        }
      },
      "hits": [
        {
          "index": "my_index",
          "id": "3",
          "score": {
            "float_value": 1.0
          },
          "source": "eyAidGl0bGUiOiAiUnVzaCIsICJ5ZWFyIjogMjAxM30=",
          "meta_fields": {}
        }
      ],
      "max_score": {
        "float_value": 1.0
      }
    }
  }
}

Java gRPC client example

The following example shows a Java client-side program that submits an example search term query gRPC request and then prints the number of hits returned in the search response:

import org.opensearch.protobufs.*;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;

public class SearchClient {
    public static void main(String[] args) {
        ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 9400)
            .usePlaintext()
            .build();

        SearchServiceGrpc.SearchServiceBlockingStub stub = SearchServiceGrpc.newBlockingStub(channel);

        // Create a term query
        TermQuery termQuery = TermQuery.newBuilder()
            .setField("director")
            .setValue(FieldValue.newBuilder().setStringValue("Nolan").build())
            .build();

        // Create query container
        QueryContainer queryContainer = QueryContainer.newBuilder()
            .setTerm(termQuery)
            .build();

        // Create search request body
        SearchRequestBody requestBody = SearchRequestBody.newBuilder()
            .setQuery(queryContainer)
            .setSize(5)
            .build();

        // Create search request
        SearchRequest request = SearchRequest.newBuilder()
            .addIndex("movies")
            .setSearchRequestBody(requestBody)
            .build();

        try {
            SearchResponse response = stub.search(request);

            // Handle the response
            System.out.println("Search took: " + response.getTook() + " ms");
            System.out.println("Timed out: " + response.getTimedOut());

            HitsMetadata hits = response.getHits();
            if (hits.hasTotal()) {
                System.out.println("Total hits: " + hits.getTotal().getTotalHits().getValue());
            }

            // Process individual hits
            for (HitsMetadataHitsInner hit : hits.getHitsList()) {
                System.out.println("Hit ID: " + hit.getXId());
                System.out.println("Hit Index: " + hit.getXIndex());
                if (hit.hasXScore()) {
                    System.out.println("Score: " + hit.getXScore().getDouble());
                }
            }
        } catch (io.grpc.StatusRuntimeException e) {
            System.err.println("gRPC search request failed with status: " + e.getStatus());
            System.err.println("Error message: " + e.getMessage());
        }

        channel.shutdown();
    }
}

Python gRPC client example

The following example shows how to send the same request using a Python client application.

First, install the opensearch-protobufs package using pip:

pip install opensearch-protobufs==0.24.0

Use the following code to send the request:

import grpc

from opensearch.protobufs.schemas import *
from opensearch.protobufs.services import SearchServiceStub


channel = grpc.insecure_channel(
    target="localhost:9400",
)

search_stub = SearchServiceStub(channel)

# Create a term query
term_query = TermQuery(
    field="field",
    value=FieldValue(string="value")
)
query_container = QueryContainer(term=term_query)

# Create a search request
request = SearchRequest(
    search_request_body=SearchRequestBody(query=query_container),
    index=["my-index"]
)

# Send request and handle response
try:
    response = search_stub.Search(request=request)
    if response.hits:
        print("Found {} hits".format(response.hits.total))
        print(response.hits)
    elif response.timed_out or response.terminated_early:
        print("Request timed out or terminated early")
    elif response.x_shards.failed:
        print("Some shards failed to execute the search")
        print(response.x_shards.failures)
except grpc.RpcError as e:
    if e.code() == StatusCode.UNAVAILABLE:
        print("Failed to reach server: {}".format(e))
    elif e.code() == StatusCode.PERMISSION_DENIED:
        print("Permission denied: {}".format(e))
    elif e.code() == StatusCode.INVALID_ARGUMENT:
        print("Invalid argument: {}".format(e))
finally:
    channel.close()