streamstats

The streamstats command calculates cumulative or rolling statistics as events that are processed in order. Unlike stats or eventstats, which operate on the entire dataset at once, streamstats processes events incrementally, making it suitable for time-series and sequence-based analysis.

Key features include support for the window (sliding window calculations) and current (whether to include the current event in calculations) parameters and specialized use cases such as identifying trends or detecting changes over sequences of events.

Comparing stats, eventstats, and streamstats

The stats, eventstats, and streamstats commands can all generate aggregations such as average, sum, and maximum. However, they differ in how they operate and the results they produce. The following table summarizes these differences.

Syntax

The streamstats command has the following syntax:

streamstats [bucket_nullable=bool] [current=<bool>] [window=<int>] [global=<bool>] [reset_before="("<eval-expression>")"] [reset_after="("<eval-expression>")"] <function>... [by-clause]

The following are examples of the streamstats command syntax:

source = table | streamstats avg(a)
source = table | streamstats current = false avg(a)
source = table | streamstats window = 5 sum(b)
source = table | streamstats current = false window = 2 max(a)
source = table | where a < 50 | streamstats count(c)
source = table | streamstats min(c), max(c) by b
source = table | streamstats count(c) as count_by by b | where count_by > 1000
source = table | streamstats dc(field) as distinct_count
source = table | streamstats distinct_count(category) by region
source = table | streamstats current=false window=2 global=false avg(a) by b
source = table | streamstats window=2 reset_before=a>31 avg(b)
source = table | streamstats current=false reset_after=a>31 avg(b) by c

Parameters

The streamstats command supports the following parameters.

Aggregation functions

The streamstats command supports the following aggregation functions:

• COUNT – Count of values
• SUM – Sum of numeric values
• AVG – Average of numeric values
• MAX – Maximum value
• MIN – Minimum value
• VAR_SAMP – Sample variance
• VAR_POP – Population variance
• STDDEV_SAMP – Sample standard deviation
• STDDEV_POP – Population standard deviation
• DISTINCT_COUNT/DC – Distinct count of values
• EARLIEST – Earliest value by timestamp
• LATEST – Latest value by timestamp

Example 1: Calculate the running average, sum, and count of a field by group

The following query calculates the running average age, running sum of age, and running count of events for all accounts, grouped by gender:

source=accounts
| streamstats avg(age) as running_avg, sum(age) as running_sum, count() as running_count by gender

The query returns the following results:

Example 2: Calculate the running maximum over a 2-row window

The following query calculates the running maximum age over a 2-row window, excluding the current event:

source=state_country
| streamstats current=false window=2 max(age) as prev_max_age

The query returns the following results:

Example 3: Global compared to group-specific windows

The global parameter takes the following values:

• true: A global window is applied across all rows, but the calculations inside the window still respect the by groups.
• false: The window itself is created per group, meaning each group receives an independent window.

The following example uses a sample index containing the following data:

The following examples calculate the running average of age across accounts by country, using a different global parameter.

When global=true, the window slides across all rows in input order, but aggregation is still computed by country. The sliding window size is 2:

source=state_country
| streamstats window=2 global=true avg(age) as running_avg by country

As a result, David and Rick are included in the same sliding window when computing running_avg across all rows globally:

In contrast, when global=false, each by group forms an independent stream and window:

source=state_country
| streamstats window=2 global=false avg(age) as running_avg by country

David and Hello form a window for the USA group. As a result, for David, the running_avg is 35.0 instead of 40.0 in the previous case:

Example 4: Conditional statistics reset

The following query calculates the running average of age across accounts by country, with resets applied:

source=state_country
| streamstats current=false reset_before=age>34 reset_after=age<25 avg(age) as avg_age by country

The query returns the following results:

Example 5: Null bucket behavior

When bucket_nullable=false, null values are excluded from group-by aggregations:

source=accounts
| streamstats bucket_nullable=false count() as cnt by employer
| fields account_number, firstname, employer, cnt

Rows in which the by field is null are excluded from aggregation, so the cnt for Dale is null:

When bucket_nullable=true, null values are treated as a valid group:

source=accounts
| streamstats bucket_nullable=true count() as cnt by employer
| fields account_number, firstname, employer, cnt

As a result, the cnt for Dale is included and calculated normally: