Composite aggregation #

Paginated multi-source bucket aggregation: the composite key is a tuple built from several sources (a terms field, a histogram interval, a date_histogram, a geotile_grid), and pages are pulled with an after cursor. It does what a single terms cannot: group by several dimensions at once, and stream all combinations without oversizing size.

How it works #

Each entry in sources is named: { "day": { … } } binds day to one dimension of the composite key. A bucket’s key maps each source name to its value; buckets come back in ascending key order — first source most significant, values compared as numbers when both are (numbers sort before strings), as strings otherwise.

There is no doc_count ordering and no truncation: every combination occurring in the matched documents is materialized, and size only slices the sorted list into pages — the full set is always reachable by cursor, where terms keeps its top-size buckets and drops the rest.

A page carries an after_key, the key of its last bucket. Pass it back as after and the next page holds only buckets sorting strictly greater. A short final page still carries an after_key; the walk ends when a page comes back empty.

Per document: a terms source over a multi-valued field yields one value per element, so one document can land in several buckets; a document missing a value for any source is dropped. histogram keys are floor(v / interval) * interval, date_histogram keys are epoch milliseconds (offset/time_zone shift boundaries), geotile_grid keys are map tiles; sub-aggregations run only for the returned page.

SQL GROUP BY … LIMIT plans lower through composite — LIMIT becomes size, one source per grouping key.

Examples #

Page through merchant combinations, five buckets at a time — the SQL side is GROUP BY … LIMIT plus the envelope’s keyset cursor:

POST /sales/_search
{
  "aggs": {
    "matrix": {
      "composite": {
        "size": 100,
        "sources": [
          { "merchant": { "terms": { "field": "merchant" } } },
          { "day": { "date_histogram": { "field": "date", "calendar_interval": "day" } } }
        ],
        "after": { "merchant": "shop_b", "day": 1700000000000 }
      }
    }
  }
}

Use cases #

• Cross-tab reports. sales crossed type × status lands in 9 of 12 possible cells (mug was never pending, sticker only ever pending); nest sum(amount) for revenue per cell.
• Export the full bucket space. terms cuts to its top size; composite walks everything — loop after until the empty page, five rows at a time over the 9 cells above.
• Deep pagination. after walks high-cardinality keys page by page.
• Time × dimension matrices. date_histogram by day plus terms on type gives one bucket per day-and-type that occurred across the 120 sales (2024-01-01 through 2025-04-21).

Parameters for composite #

• size
  (Optional, integer, default: 10) Maximum number of composite buckets to return per page.
• sources
  (Required, array) Named sources, one per dimension of the composite key, as { "<name>": { "<type>": { … } } }; <type> is one of:
  • terms — distinct values of a field (an order is accepted; keys always come back ascending).
  • histogram — numeric buckets of a fixed numeric interval.
  • date_histogram — calendar_interval (minute…year) or fixed_interval (e.g. "30m"; the legacy interval still works), plus offset, time_zone, format. Keys are epoch milliseconds.
  • geotile_grid — map tiles at precision (default 7, max 29).
• after
  (Optional, object) Pagination cursor — the previous page’s after_key.
• aggs
  (Optional, object) Nested sub-aggregations, computed per bucket.

Related #

• terms — top-N buckets by doc_count, no cursor
• multi_terms — tuple keys from several fields
• date_histogram — the time source, standalone
• geotile_grid — the map-tile source, standalone