Lucene expression language

The expression language compiles a single JavaScript expression into JVM bytecode. It evaluates numeric formulas over doc values and supports no statements, variable declarations, or string handling.

An expression is resolved to direct doc values accessors at compile time, so evaluating it for one document reduces to arithmetic, with no type resolution, map lookups, or method dispatch. A formula that both the Lucene expression language and Painless can express typically runs faster as an expression than as a Painless script.

The language is sandboxed and enabled for inline and stored scripts by default, and it can be used in the score, field, filter, number_sort, aggs, bucket_aggregation, aggregation_selector, and terms_set contexts. Set lang to expression to select it.

Syntax

An expression script consists of a single expression that evaluates to a number, and the conditional ?: operator is its only form of control flow. An expression can use the arithmetic, comparison, and bitwise operators of JavaScript along with the functions in the Lucene expressions module, including abs, min, max, sqrt, pow, log, the trigonometric functions, and haversin for geographic distance.

An expression can reference the following values:

• A document field, through doc['field_name'].value.
• A property or method of a field, such as doc['field_name'].empty or doc['field_name'].sum().
• A script parameter, by its name alone. An expression references a parameter as discount, not as params.discount; using the params prefix fails with Unknown variable [params].
• The relevance score, _score, which is available only in a script_score context.

Referencing a parameter without the params prefix is the syntax difference most likely to be overlooked when a script is moved from Painless to expression.

Numeric field API

The following table lists the properties and methods available on a numeric field.

A field that is absent from the document evaluates to 0. To substitute a different value, test empty first, as in doc['ratings'].empty ? 3 : doc['ratings'].value.

A multi-valued field evaluates to its smallest value. To select a different one, call the corresponding method, as in doc['ratings'].sum().

Boolean fields are exposed as numbers, with true as 1 and false as 0, so a Boolean can gate a calculation directly: doc['on_sale'].value ? doc['price'].value - doc['price'].value * discount : doc['price'].value.

The following search returns each product’s mean rating, substituting -1 for the one product that has no ratings. It uses the scripting-products index, which is created in Test setup:

GET scripting-products/_search
{
  "_source": ["name"],
  "sort": [{ "sku": "asc" }],
  "script_fields": {
    "average_rating": {
      "script": {
        "lang": "expression",
        "source": "doc['ratings'].empty ? unrated : doc['ratings'].avg()",
        "params": { "unrated": -1 }
      }
    }
  }
}

The ultrawide monitor, which has no ratings values, returns the substitute value:

{
  "took": 71,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "scripting-products",
        "_id": "3",
        "_score": null,
        "_source": {
          "name": "Mechanical keyboard"
        },
        "fields": {
          "average_rating": [
            5.0
          ]
        },
        "sort": [
          "ACC-2001"
        ]
      },
      {
        "_index": "scripting-products",
        "_id": "1",
        "_score": null,
        "_source": {
          "name": "Wireless noise cancelling headphones"
        },
        "fields": {
          "average_rating": [
            4.25
          ]
        },
        "sort": [
          "AUD-1001"
        ]
      },
      {
        "_index": "scripting-products",
        "_id": "2",
        "_score": null,
        "_source": {
          "name": "Wireless earbuds"
        },
        "fields": {
          "average_rating": [
            4.0
          ]
        },
        "sort": [
          "AUD-1002"
        ]
      },
      {
        "_index": "scripting-products",
        "_id": "4",
        "_score": null,
        "_source": {
          "name": "Ultrawide monitor"
        },
        "fields": {
          "average_rating": [
            -1.0
          ]
        },
        "sort": [
          "DSP-3001"
        ]
      }
    ]
  }
}

Every value that an expression returns is a double, which is why the ratings appear as 5.0 and -1.0 rather than as integers.

Date field API

A date field is exposed as a number of milliseconds elapsed since the Unix epoch, so the entire numeric field API applies to it. In addition, the date property gives access to individual calendar components.

The following table lists the components available on a date field.

To find the number of whole years between two date fields, subtract their years: doc['release_date'].date.year - doc['discontinued_date'].date.year.

The geopoint field API

The following table lists the properties available on a geo_point field.

Combine these with haversin to compute a great-circle distance in kilometers. The following search orders products by how far their warehouse is from Washington, DC, with the origin supplied as parameters so that the same compiled script serves every user location:

GET scripting-products/_search
{
  "_source": ["name", "warehouse"],
  "sort": {
    "_script": {
      "type": "number",
      "script": {
        "lang": "expression",
        "source": "haversin(lat, lon, doc['warehouse'].lat, doc['warehouse'].lon)",
        "params": { "lat": 38.9072, "lon": -77.0369 }
      },
      "order": "asc"
    }
  }
}

The sort value on each result is the computed distance in kilometers:

{
  "took": 12,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "scripting-products",
        "_id": "3",
        "_score": null,
        "_source": {
          "name": "Mechanical keyboard",
          "warehouse": { "lon": -87.6298, "lat": 41.8781 }
        },
        "sort": [
          955.1850203827704
        ]
      },
      {
        "_index": "scripting-products",
        "_id": "4",
        "_score": null,
        "_source": {
          "name": "Ultrawide monitor",
          "warehouse": { "lon": -97.7431, "lat": 30.2672 }
        },
        "sort": [
          2118.1759666556795
        ]
      },
      {
        "_index": "scripting-products",
        "_id": "1",
        "_score": null,
        "_source": {
          "name": "Wireless noise cancelling headphones",
          "warehouse": { "lon": -122.3321, "lat": 47.6062 }
        },
        "sort": [
          3736.260216407672
        ]
      },
      {
        "_index": "scripting-products",
        "_id": "2",
        "_score": null,
        "_source": {
          "name": "Wireless earbuds",
          "warehouse": { "lon": -122.4194, "lat": 37.7749 }
        },
        "sort": [
          3918.550756602828
        ]
      }
    ]
  }
}

Limitations

The expression language has the following limitations:

• Only numeric, Boolean, date, and geo_point fields are readable, so an expression cannot compare or concatenate strings. Reading a keyword field fails with a link error caused by Field [sku] must be numeric, date, or geopoint. Enabling fielddata on a text field makes it readable in Painless but not in an expression, which reports the same error.
• Stored fields and the _source are unavailable, so an expression cannot read a value that is not in doc values. Both are reached through params, which an expression does not receive, so params._source and params._fields fail with Unknown variable [params].
• There is no way to test whether a field exists in the mapping. doc['field'].empty reports only whether a mapped field has a value in the current document. Referencing an unmapped field fails with a link error caused by Field [discount] does not exist in mappings, and wrapping the reference in empty produces the same failure.
• Every result is a double, so an expression cannot return a string, a Boolean, or a structured value.

When a formula needs any of these, use Painless instead.

Related documentation

• Painless scripting language
• Accessing document fields in scripts
• How to use scripts