Missing aggregation #

A single-bucket aggregation of the documents that have no indexed value for a field — the inverse of the exists query. It turns data gaps into a number you can alert on, instead of discovering them when a report quietly undercounts.

How it works #

For every document the query matched, the engine checks whether the field has an indexed value in the column store — an absent field and an explicit null both count as missing. The matching documents form one bucket shaped { "doc_count": N, …sub-aggs }; sub-aggregations run over exactly those incomplete documents, so you can ask not just how many rows are broken but how they break.

Examples #

The teaching dataset’s real gap sits in articles: 24 of its 44 documents carry no author. Count them live:

{
  ...
  "aggregations": {
    "no_author": {
      "doc_count": 24
    }
  }
}

Note that the count is scoped to the query: add a query and missing reports the gap among the MATCHES only, which is what makes it usable as a per-slice quality signal rather than a global one:

POST /articles/_search
{
  "size": 0,
  "query": { "term": { "field": "tags", "value": "search" } },
  "aggs": {
    "no_author": { "missing": { "field": "author" } }
  }
}

Use cases #

• Data-quality dashboards. Track the author gap (24 of 44 documents today) as a trend line; a rising count means an upstream producer stopped filling the field.
• Coverage audits. Run one missing per required field to grade completeness before turning a dataset over to analysts.
• Backfill targeting. Nest a terms on another field inside the bucket to learn which slice of the data is incomplete, then aim the repair job there.
• Alerting. Pair the count with the exists query coverage ratio for a cheap SLO on ingestion hygiene.
• Schema migrations. Before promoting an optional field to required, check how many existing documents would have failed the stricter mapping.

Parameters for missing #

• field
  (Required, string) Field to check for missing values.
• aggs
  (Optional, object) Nested sub-aggregations, computed over the documents missing the field.

Related #

• exists query — the row-level counterpart: match documents that have the field
• terms — its missing parameter routes gap documents into a named bucket instead of dropping them
• global — the other scope-carving single bucket