Significant terms aggregation #

A multi-bucket aggregation returning terms that are unusually frequent in the current scope (foreground) compared with the whole index (background) — the classic “what’s special about this subset” query.

In this engine significant_terms currently approximates that idea by foreground frequency: it ranks the scope’s values by their in-scope doc_count, without computing background statistics. Nest it under a query or a parent bucket to scope the foreground set; the values that dominate that scope come out on top.

How it works #

Bucketing follows terms: one bucket per unique value of field, include/exclude filter candidate keys (exact-value arrays or a single wildcard string), and min_doc_count (default 1) drops thin buckets. Buckets are ordered by doc_count descending and truncated to size (default 10).

Two differences from a plain terms are worth knowing:

• Buckets carry key and doc_count only — the score and bg_count fields are not part of the response, and there is no background_filter parameter.
• The significance logic lives in the engine’s significant_text sibling (below), which scores analyzed text foreground-vs-background with the JLH formula; significant_terms over keyword fields ranks by foreground count instead.

Because ordering is by in-scope count, the aggregation is most informative when the foreground is skewed: nest it inside a terms bucket (values of a class, category, or status) and compare which inner values lead each outer bucket.

Example #

Which merchants over-index inside pending sales?

All four merchants sell 10 pending items each in the evenly balanced teaching data, so the ranking is flat; skew your own data and the merchants concentrating in this status surface first.

Use cases #

• Anomaly-associated values. Inside a logs scope of error messages, which host or url values lead — the suspects for any incident.
• Per-category fingerprints. Under sales.terms(type), a nested significant_terms on merchant shows which merchant defines each product type.
• Cohort contrast. Query the cohort (pending customers, gold tier), then let significant_terms list the values that fill it.
• Sampled significance. On very large scopes, wrap the scope in a sampler first, then rank — the sample-then-signify pattern.

Parameters for significant_terms #

• field
  (Required, string) Keyword field you wish to aggregate.
• size
  (Optional, integer, default: 10) Maximum number of buckets.
• min_doc_count
  (Optional, integer, default: 1) Minimum foreground count per term.
• include / exclude
  (Optional) Restrict candidate terms (exact values or wildcard patterns).
• aggs
  (Optional, object) Nested sub-aggregations, computed per bucket.

Related #

• terms — plain frequency ranking
• rare_terms — the low-frequency tail
• sampler — cut the foreground down before ranking