Link Search Menu Expand Document Documentation Menu
Documentation

You're viewing version 3.0 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.

Derivative aggregations

The derivative aggregation is a parent aggregation used to calculate first-order and second-order derivatives of each bucket of an aggregation. “First-order derivative” and “second-order derivative” are often shortened to “first derivative” and “second derivative,” respectively. This page uses the shortened terms.

For an ordered series of buckets, derivative approximates a first derivative as the difference between metric values in the current and previous buckets.

Parameters

The derivative aggregation takes the following parameters.

Parameter Required/Optional Data type Description
buckets_path Required String The path of the aggregation buckets to be aggregated. See Buckets path.
gap_policy Optional String The policy to apply to missing data. Valid values are skip and insert_zeros. Default is skip. See Data gaps.
format Optional String A DecimalFormat formatting string. Returns the formatted output in the aggregation’s value_as_string property.

Example: First derivative

The following example creates a date histogram with a one-month interval from the OpenSearch Dashboards e-commerce sample data. The sum sub-aggregation calculates the sum of all bytes for each month. Finally, the derivative aggregation calculates the first derivative of the sum sub-aggregation. The first derivative is estimated as the difference between the number of bytes in the current month and the previous month:

GET opensearch_dashboards_sample_data_logs/_search
{
  "size": 0,
  "aggs": {
    "sales_per_month": {
      "date_histogram": {
        "field": "@timestamp",
        "calendar_interval": "month"
      },
      "aggs": {
        "number_of_bytes": {
          "sum": {
            "field": "bytes"
          }
        },
        "bytes_deriv": {
          "derivative": {
            "buckets_path": "number_of_bytes"
          }
        }
      }
    }
  }
}

Example response: First derivative

The response shows derivatives computed for the second and third buckets:

{
  "took": 6,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "sales_per_month": {
      "buckets": [
        {
          "key_as_string": "2025-03-01T00:00:00.000Z",
          "key": 1740787200000,
          "doc_count": 480,
          "number_of_bytes": {
            "value": 2804103
          }
        },
        {
          "key_as_string": "2025-04-01T00:00:00.000Z",
          "key": 1743465600000,
          "doc_count": 6849,
          "number_of_bytes": {
            "value": 39103067
          },
          "bytes_deriv": {
            "value": 36298964
          }
        },
        {
          "key_as_string": "2025-05-01T00:00:00.000Z",
          "key": 1746057600000,
          "doc_count": 6745,
          "number_of_bytes": {
            "value": 37818519
          },
          "bytes_deriv": {
            "value": -1284548
          }
        }
      ]
    }
  }
}

No derivative is calculated for the first bucket because no previous bucket is available for that bucket.

Example: Second derivative

To calculate a second derivative, chain one derivative aggregation to another:

GET opensearch_dashboards_sample_data_logs/_search
{
  "size": 0,
  "aggs": {
    "sales_per_month": {
      "date_histogram": {
        "field": "@timestamp",
        "calendar_interval": "month"
      },
      "aggs": {
        "number_of_bytes": {
          "sum": {
            "field": "bytes"
          }
        },
        "bytes_1st_deriv": {
          "derivative": {
            "buckets_path": "number_of_bytes"
          }
        },
        "bytes_2nd_deriv": {
          "derivative": {
            "buckets_path": "bytes_1st_deriv"
          }
        }
      }
    }
  }
}

Example response: Second derivative

{
  "took": 6,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "sales_per_month": {
      "buckets": [
        {
          "key_as_string": "2025-03-01T00:00:00.000Z",
          "key": 1740787200000,
          "doc_count": 480,
          "number_of_bytes": {
            "value": 2804103
          }
        },
        {
          "key_as_string": "2025-04-01T00:00:00.000Z",
          "key": 1743465600000,
          "doc_count": 6849,
          "number_of_bytes": {
            "value": 39103067
          },
          "bytes_1st_deriv": {
            "value": 36298964
          }
        },
        {
          "key_as_string": "2025-05-01T00:00:00.000Z",
          "key": 1746057600000,
          "doc_count": 6745,
          "number_of_bytes": {
            "value": 37818519
          },
          "bytes_1st_deriv": {
            "value": -1284548
          },
          "bytes_2nd_deriv": {
            "value": -37583512
          }
        }
      ]
    }
  }
}

No first derivative is calculated for the first bucket because no previous bucket is available for that bucket. Similarly, no second derivative is calculated for the first or second buckets.

350 characters left

Have a question? .

Want to contribute? or .