kr7shnasomani/kairos / .agents/skills/elasticsearch-esql
elasticsearch-esql
FlaggedExecute ES|QL (Elasticsearch Query Language) queries, use when the user wants to query Elasticsearch data, analyze logs, aggregate metrics, explore data, or create charts and dashboards from ES|QL results.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 18.9 KB |
| package.json | manifest | 256 B |
| references/dsl-to-esql-migration.md | markdown | 19.6 KB |
| references/environment-setup.md | markdown | 2.3 KB |
| references/esql-reference.md | markdown | 70.9 KB |
| references/esql-search-strategy.md | markdown | 10.5 KB |
| references/esql-search.md | markdown | 24.7 KB |
| references/esql-version-history.md | markdown | 26.9 KB |
| references/generation-tips.md | markdown | 24.2 KB |
| references/promql-command.md | markdown | 15.1 KB |
| references/query-patterns.md | markdown | 14.7 KB |
| references/time-series-queries.md | markdown | 24.1 KB |
| scripts/esql.jsexecutable | script | 21.4 KB |
references/esql-search-strategy.md
View on GitHub# ES|QL Relevance Search Strategy
This guide teaches an agent how to perform high-quality relevance search on content indices using ES|QL.
## Table of Contents
- [Scope](#scope)
- [Quick Strategy Rules](#quick-strategy-rules)
- [Standard Workflow](#standard-workflow)
- [Multi-Stage Retrieval Pattern](#multi-stage-retrieval-pattern)
- [Retrieval Strategies](#retrieval-strategies)
- [Lexical Retrieval](#lexical-retrieval)
- [Semantic Retrieval](#semantic-retrieval)
- [Vector Retrieval](#vector-retrieval)
- [Hybrid Retrieval](#hybrid-retrieval)
- [Semantic Intent Without Semantic Field](#semantic-intent-without-semantic-field)
- [Reranking Stage](#reranking-stage)
- [Semantic Reranking](#semantic-reranking)
- [Embedding Similarity Rescore](#embedding-similarity-rescore)
- [Phrase Search](#phrase-search)
- [Multi-Index Search](#multi-index-search)
- [Weak Result Recovery](#weak-result-recovery)
- [Mandatory Rules](#mandatory-rules)
- [Final Decision Process](#final-decision-process)
## Scope
This guidance is designed for:
- document search
- knowledge bases
- articles
- documentation
- product content
It is **not** intended for logs or observability datasets.
Agents should always follow a **multi-stage retrieval pattern**:
```text
retrieve → (optional fuse) → rerank
```
Start with inexpensive retrieval and apply expensive ranking only to a limited candidate set.
## Quick Strategy Rules
| Situation | Strategy |
| ------------------------------------- | ----------------------------------- |
| `semantic_text` field exists | semantic `MATCH` |
| only `text` fields exist | lexical `MATCH` |
| `dense_vector` field exists | `KNN` retrieval |
| lexical + semantic fields exist | hybrid retrieval |
| semantic intent but no semantic field | lexical → `RERANK` |
| exact wording required | `MATCH_PHRASE` |
| multiple indices | branch search using `FORK` + `FUSE` |
## Standard Workflow
### 1. Identify Indices
Use the narrowest index pattern that satisfies the request.
```esql
// Single index
FROM knowledge-base
// Multiple indices
FROM docs-*,articles-*
```
### 2. Inspect Mappings
Before writing queries, identify searchable fields.
Preferred fields for content search:
```text
title
name
subject
summary
body
content
description
text
```
### Field Types
| Field type | Purpose |
| --------------- | ------------------ |
| `semantic_text` | semantic retrieval |
| `text` | lexical retrieval |
| `dense_vector` | vector retrieval |
| `keyword` | filtering only |
**Never** use `keyword` fields for natural language search.
### Preferred Field Ranking
1. `title`
2. `summary`
3. `body` / `content`
4. `description`
5. `text`
Short fields provide **precision**. Long fields provide **recall**.
### 3. Choose Retrieval Strategy
Use the [Quick Strategy Rules](#quick-strategy-rules) table to select the right approach based on the available field
types. Then follow the matching retrieval pattern below.
## Multi-Stage Retrieval Pattern
Always follow this structure:
1. retrieve candidate documents
2. optionally combine retrieval strategies
3. rerank candidates
Typical candidate size: **50–200 documents**. Use smaller sets when fields are strong. Use larger sets when recall is
important.
## Retrieval Strategies
### Lexical Retrieval
Use when only `text` fields exist.
```esql
FROM my-index METADATA _score
| WHERE MATCH(title, ?query) OR MATCH(body, ?query)
| SORT _score DESC
| LIMIT 100
```
Guidelines:
- **one field per `MATCH`** — the first argument is a **single mapped field** (`title`, `content`), not a quoted
pseudo-field like `"title,content"` and not two identifiers before the query (`MATCH(title, body, "q")` is invalid);
use `MATCH(a, ?query) OR MATCH(b, ?query)` or `QSTR` / `KQL` for multi-field search
- a single `MATCH` on the main content field is often sufficient
- add additional fields (title, summary) only when recall is poor or the user explicitly asks for broader search
- title fields provide precision, body fields provide recall
- prefer `MATCH` over `LIKE` or `RLIKE`
### Semantic Retrieval
Use when a `semantic_text` field exists. `MATCH` is **required** for `semantic_text` fields — it automatically performs
vector-based semantic search. No separate function is needed.
```esql
FROM my-index METADATA _score
| WHERE MATCH(semantic_body, ?query)
| SORT _score DESC
| LIMIT 100
```
Prefer the semantic field representing the **main document body**.
### Vector Retrieval
> **Version:** `KNN` is 9.2+ (preview). `TEXT_EMBEDDING` is 9.3+. Verify cluster version via `esql-version-history.md`
> before using these functions. For clusters below 9.2, use semantic retrieval with `semantic_text` + `MATCH` instead.
Use when embeddings are stored as `dense_vector`. `KNN` can also target `semantic_text` fields.
```esql
FROM my-index METADATA _score
| WHERE KNN(content_embedding, TEXT_EMBEDDING(?query, "embedding_endpoint"))
| SORT _score DESC
| LIMIT 100
```
Rules:
- the query embedding model must match the document embeddings
- always retrieve a bounded candidate set
- avoid embedding operations across the full index
### Hybrid Retrieval
> **Version:** `FORK` is 8.19/9.1+ (preview). `FUSE` is 9.2+ (preview). On clusters below 9.2, use lexical retrieval
> followed by `RERANK` as a fallback. On clusters below 8.19/9.1, use a single-branch `MATCH` with `RERANK`.
Use when both lexical and semantic fields exist.
```esql
FROM my-index METADATA _id, _index, _score
| FORK
(
WHERE MATCH(title, ?query) OR MATCH(body, ?query)
| SORT _score DESC
| LIMIT 100
)
(
WHERE MATCH(semantic_body, ?query)
| SORT _score DESC
| LIMIT 100
)
| FUSE
| SORT _score DESC
| LIMIT 100
```
Hybrid retrieval improves both **recall and precision**.
Pipeline:
```text
retrieve lexically
retrieve semantically
fuse
rerank
```
## Semantic Intent Without Semantic Field
If semantic search is requested but the index lacks `semantic_text`:
1. retrieve candidates lexically
2. rerank results semantically
```esql
FROM my-index METADATA _score
| WHERE MATCH(title, ?query) OR MATCH(body, ?query) OR MATCH(summary, ?query)
| SORT _score DESC
| LIMIT 100
```
**Never** stop at "semantic search unavailable" without attempting lexical retrieval.
## Reranking Stage
> **Version:** `RERANK` is 9.2+ (preview). On clusters below 9.2, skip the reranking stage and rely on initial retrieval
> scoring. For clusters below 9.2, sorting by `_score DESC` after `MATCH` provides BM25 or vector-based ordering.
Always rerank a bounded candidate set.
### Semantic Reranking
```esql
FROM my-index METADATA _score
| WHERE MATCH(body, ?query)
| SORT _score DESC
| LIMIT 100
| RERANK body ON ?query
| SORT _score DESC
| LIMIT 20
```
Use when:
- lexical retrieval produced good candidates
- semantic ranking improves ordering
### Embedding Similarity Rescore
> **Version:** `V_COSINE` and other vector similarity functions are 9.3+ (preview). Verify cluster version via
> `esql-version-history.md` before using these functions.
Use when document embeddings exist.
```esql
FROM my-index METADATA _score
| WHERE MATCH(body, ?query)
| SORT _score DESC
| LIMIT 100
| EVAL q = TEXT_EMBEDDING(?query, "embedding_endpoint")
| EVAL semantic_score = V_COSINE(content_embedding, q)
| SORT semantic_score DESC
| LIMIT 20
```
Rules:
- compute the query embedding once
- compare only against candidate documents
- avoid vector scans across entire indices
## Phrase Search
Use when exact wording matters.
```esql
FROM my-index METADATA _score
| WHERE MATCH_PHRASE(body, ?phrase)
| SORT _score DESC
| LIMIT 20
```
Typical cases:
- product names
- quoted text
- error messages
- legal phrases
## Multi-Index Search
When querying multiple indices:
1. inspect mappings
2. confirm compatible fields
3. branch queries when schemas differ
4. fuse results
5. rerank candidates
> **`_index` is not implicit:** You may use `_index` in queries (for example `WHERE _index LIKE "docs-%"`) **only** if
> the `FROM` clause requests it via `METADATA _index` (typically `METADATA _id, _index, _score` for `FORK`/`FUSE`). If
> `_index` is omitted from `METADATA`, the column does not exist in the pipeline — the query fails with an unknown-field
> error. Prefer the [compatible-schemas](#compatible-schemas) pattern when you do not need per-index branching.
### Compatible Schemas
```esql
FROM docs-*,articles-* METADATA _score
| WHERE MATCH(title, ?query) OR MATCH(body, ?query)
| SORT _score DESC
| LIMIT 20
```
### Different Schemas
```esql
FROM docs-*,support-* METADATA _id, _index, _score
| FORK
(
WHERE _index LIKE "docs-%"
| WHERE MATCH(body, ?query)
| SORT _score DESC
| LIMIT 100
)
(
WHERE _index LIKE "support-%"
| WHERE MATCH(content, ?query)
| SORT _score DESC
| LIMIT 100
)
| FUSE
| SORT _score DESC
| LIMIT 20
```
Always keep the pattern simple:
```text
retrieve per index family → fuse → rerank
```
## Weak Result Recovery
If results are weak:
1. search additional content fields
2. increase candidate size
3. apply semantic reranking
4. switch to hybrid retrieval
5. inspect mappings for stronger fields
6. split multi-index search by index family
Avoid jumping directly to expensive ranking.
## Mandatory Rules
**Always:**
- inspect mappings first
- retrieve candidates before reranking
- limit candidate sets to ~50–200
- use `_score` for ranking
- prefer content fields over metadata
- follow retrieve → fuse → rerank
**Never:**
- search natural language in `keyword` fields
- run embedding operations across entire indices
- skip the retrieval stage
- use `LIKE` for relevance search
- dump full mappings unless necessary
## Default Query Patterns
Use the examples from [Retrieval Strategies](#retrieval-strategies) as starting templates:
- **Lexical** — see [Lexical Retrieval](#lexical-retrieval)
- **Semantic** — see [Semantic Retrieval](#semantic-retrieval)
- **Hybrid** — see [Hybrid Retrieval](#hybrid-retrieval), then add a [Reranking Stage](#reranking-stage)
Use the hybrid pattern when both lexical and semantic fields exist.
## Final Decision Process
Follow this order:
1. inspect mappings
2. choose best retrieval field family
3. retrieve candidates
4. fuse if necessary
5. rerank
6. return results
Mental model:
```text
retrieve → fuse → rerank
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |