Percentiles aggregation #

A multi-value metrics aggregation that calculates one or more percentiles over numeric values extracted from the aggregated documents.

Percentiles show the point at which a certain percentage of observed values occur. For example, the 95th percentile is the value which is greater than 95% of the observed values.

Percentiles are often used to find outliers. In normal distributions, the 0.13th and 99.87th percentiles represents three standard deviations from the mean. Any data which falls outside three standard deviations is often considered an anomaly.

When a range of percentiles are retrieved, they can be used to estimate the data distribution and determine if the data is skewed, bimodal, etc.

Examples #

Assume your data consists of website load times. The average and median load times are not overly useful to an administrator. The max may be interesting, but it can be easily skewed by a single slow response.

Let’s look at a range of percentiles representing load time:

POST latency/_search
{
  "aggs": {
    "load_time_outlier": {
      "percentiles": {
        "field": "load_time"
      }
    }
  }
}

By default, the percentile metric will generate a range of percentiles: [1, 5, 25, 50, 75, 95, 99]. The response will look like this:

{
  ...
 "aggregations": {
    "load_time_outlier": {
      "values": {
        "1.0": 10.0,
        "5.0": 30.0,
        "25.0": 170.0,
        "50.0": 445.0,
        "75.0": 720.0,
        "95.0": 940.0,
        "99.0": 980.0
      }
    }
  }
}

As you can see, the aggregation will return a calculated value for each percentile in the default range. If we assume response times are in milliseconds, it is immediately obvious that the webpage normally loads in 10-725ms, but occasionally spikes to 945-985ms.

Often, administrators are only interested in outliers — the extreme percentiles. We can specify just the percents we are interested in (requested percentiles must be a value between 0-100 inclusive):

POST latency/_search
{
  "aggs": {
    "load_time_outlier": {
      "percentiles": {
        "field": "load_time",
        "percents": [95, 99, 99.9]
      }
    }
  }
}

Parameters for avg #

• field
  (Required, string) Field you wish to aggregate.
• percents
  (Optional, array) A range of percentiles that are calculated. Default is [1, 5, 25, 50, 75, 95, 99].

keyed #

By default the keyed flag is set to true which associates a unique string key with each bucket and returns the ranges as a hash rather than an array. Setting the keyed flag to false will disable this behavior:

POST latency/_search
{
  "aggs": {
    "load_time_outlier": {
      "percentiles": {
        "field": "load_time",
        "keyed": false
      }
    }
  }
}

Response:

{
  ...
  "aggregations": {
    "load_time_outlier": {
      "values": [
        {
          "key": 1.0,
          "value": 10.0
        },
        {
          "key": 5.0,
          "value": 30.0
        },
        {
          "key": 25.0,
          "value": 170.0
        },
        {
          "key": 50.0,
          "value": 445.0
        },
        {
          "key": 75.0,
          "value": 720.0
        },
        {
          "key": 95.0,
          "value": 940.0
        },
        {
          "key": 99.0,
          "value": 980.0
        }
      ]
    }
  }
}