https://pizza.rs/tags/analyzer/Recent content in Analyzer on INFINI PizzaHugoenhttps://pizza.rs/docs/references/search/analysis/analysis/Mon, 01 Jan 0001 00:00:00 +0000https://pizza.rs/docs/references/search/analysis/analysis/<h1 id="analyze-text"> Analyze text <a class="anchor" href="#analyze-text">#</a> </h1> <p>The analysis workbench: run the server&rsquo;s tokenizers, normalizers, token filters and analyzers over sample text and see the exact token stream — the same components the indexing path uses, so what you debug here is exactly what collections run. The analysis chain decides what is searchable: query and document must reduce to the same tokens.</p> <h2 id="how-it-works"> How it works <a class="anchor" href="#how-it-works">#</a> </h2> <p>Every analyzer is assembled from three stages, run in this order:</p>https://pizza.rs/docs/references/search/analysis/analysis-builtins/Mon, 01 Jan 0001 00:00:00 +0000https://pizza.rs/docs/references/search/analysis/analysis-builtins/<h1 id="built-in-analysis-components"> Built-in analysis components <a class="anchor" href="#built-in-analysis-components">#</a> </h1> <p>The server ships <strong>83</strong> analyzers, <strong>50</strong> tokenizers, <strong>15</strong> normalizers and <strong>327</strong> token filters, registered at startup. The authoritative, always-current list on YOUR node is:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">GET /_analysis/components </span></span></code></pre></div><p>Components are referenced by name — in a field&rsquo;s schema (<code>&quot;analyzer&quot;: &quot;english&quot;</code>), in a <a href="https://pizza.rs/docs/references/collection/create/">custom analyzer definition</a>, or in an ad-hoc <a href="https://pizza.rs/docs/references/search/analysis/analysis/"><code>_analysis/analyze</code></a> pipeline.</p> <h2 id="how-it-works"> How it works <a class="anchor" href="#how-it-works">#</a> </h2> <p>The catalog is assembled in two layers. The engine core registers the minimal set that is always present: the <code>standard</code> and <code>whitespace</code> tokenizers, the <code>lowercase</code> normalizer, the <code>lowercase</code>/<code>uppercase</code>/ <code>stopwords</code> token filters, and the two named analyzers composed from them (<code>standard</code>, <code>whitespace</code>). Every other name below comes from the ~40 analysis plugin crates under <code>contrib/analysis-*</code>, registered at startup in one pass — foundation plugins first (<code>core</code>, <code>stemmers</code>, <code>icu</code>), then the CJK and per-language plugins, which intentionally override the basic analyzers from <code>core</code> with richer pipelines (language stop words, normalization, stemming).</p>