topoteretes/cognee / .agents/skills/cognee-performance
cognee-performance
VerifiedUse when cognee is slow, expensive, or hitting rate limits — speeding up or throttling ingestion (remember/cognify batching, chunk size, LLM and embedding rate limits, per-stage models), cutting read latency, estimating cost before ingesting, running work in the background, or planning how to scale a deployment.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.9 KB |
SKILL.md
View on GitHub---
name: cognee-performance
description: Use when cognee is slow, expensive, or hitting rate limits — speeding up or throttling ingestion (remember/cognify batching, chunk size, LLM and embedding rate limits, per-stage models), cutting read latency, estimating cost before ingesting, running work in the background, or planning how to scale a deployment.
---
# Tune cognee's performance
**The LLM is the bottleneck.** Building the graph costs two LLM calls per
chunk (graph extraction and summarization), and almost every other tuning
question comes down to how many of those calls run, how fast, and on which
model. Database and embedding work is small next to that.
## Use it
### 1. Estimate before you ingest
```python
estimate = await cognee.remember("./docs", dry_run=True)
print(estimate) # per-stage token counts and approximate cost; no LLM calls
```
The estimate covers graph extraction and summarization only, not
`improve()` or embeddings. Not available with
GLiNER or a remote instance.
### 2. Ingestion knobs
All are `remember()` arguments (and `cognify()` ones):
| Knob | Default | What it controls |
|---|---|---|
| `data_per_batch` | 20 | How many documents are processed **at the same time** (a concurrency limit, not a batch). Lower it to reduce load; raise it for many small files. |
| `chunks_per_batch` | 2000 (env `CHUNKS_PER_BATCH`) | How many chunks each extraction/storage call receives. Lower it to spread load and fail smaller. The DLT pipeline defaults to 100. |
| `chunk_size` | min(embedding max tokens, LLM max tokens / 2); about 8191 tokens with default OpenAI models | Max tokens per chunk. Fewer, bigger chunks mean fewer LLM calls; smaller chunks mean finer-grained graphs. |
| `run_in_background` | `False` | Return immediately (`status="running"`); `await result` later. |
| `self_improvement` | `True` | `improve()` after the graph is built. `False` (or `IMPROVE_AUTO_ENABLED=false`) skips that extra work. |
Worst-case concurrent LLM calls are about
`data_per_batch × min(chunks per document, chunks_per_batch) × 2`, and there
is no separate concurrency cap. The rate limiter below is the brake.
Several datasets in one `cognify(datasets=[...])` call run one after
another, not in parallel.
### 3. LLM throttling and models
| Env var | Default | Notes |
|---|---|---|
| `LLM_RATE_LIMIT_ENABLED` | `false` | Turn on a requests-per-interval limit |
| `LLM_RATE_LIMIT_REQUESTS` / `LLM_RATE_LIMIT_INTERVAL` | 60 / 60 s | 10 requests for local providers (Ollama, llama.cpp, LM Studio) |
| `AUTO_RATE_LIMIT` | `true` | On a 429/503/529 or timeout, switches the limiter on for 15 minutes (extended while errors continue) |
| `LLM_EXTRACTION_MODEL` / `_PROVIDER` / `_ENDPOINT` / `_API_KEY` / `_API_VERSION` | the main model | Use a cheaper or faster model just for graph extraction |
| `LLM_SUMMARIZATION_*`, `LLM_QUERY_*` | the main model | Same, for summaries and for answering queries |
| `LLM_MAX_COMPLETION_TOKENS` | 16384 | Also caps the default chunk size (half of it) |
| `LLM_ARGS` | — | Extra litellm arguments merged into every call, e.g. a request `timeout` |
Structured-output calls retry with exponential backoff for at least two
attempts and four minutes before failing; auth, not-found, and quota errors
are not retried.
The rate limiter is built from the main model's settings. A stage routed to
a local model does not get the local default of 10 requests; set
`LLM_RATE_LIMIT_REQUESTS` yourself.
### 4. Embeddings
| Env var | Default |
|---|---|
| `EMBEDDING_BATCH_SIZE` | 32 texts per request |
| `EMBEDDING_MAX_CONCURRENT_DATA_POINTS` | 150 (so 150 / 32 = 4 concurrent requests) |
| `VECTOR_WRITE_BATCH_SIZE` | 128 rows per vector write, embedded in 32-text requests under the shared request concurrency limit |
| `EMBEDDING_RATE_LIMIT_ENABLED` / `_REQUESTS` / `_INTERVAL` | off / 60 / 60 s |
Embedding calls retry transient errors, rate-limit 429s included, with
exponential jitter for up to 128 s (budget-exhaustion errors are terminal,
not retried), but `AUTO_RATE_LIMIT` does not apply to them; enable
`EMBEDDING_RATE_LIMIT_ENABLED` yourself for sustained load. Without any
credentials cognee embeds locally with fastembed (`BAAI/bge-small-en-v1.5`),
which runs on the CPU and blocks while it works.
### 5. Skip the LLM for extraction
`remember(data, extractor="gliner")` builds the graph and summaries with a
local GLiNER model: no LLM calls, embeddings still run. The runtime installs
on first use (`GLINER_AUTO_INSTALL=false` to disable; then `pip install
"cognee[gliner]"`). See the `cognee-ingestion` skill for its limits.
### 6. Read latency
- **`AUTO_FEEDBACK=false`** is the biggest win for chat-style use: by default
every answered turn (the default session is used when no `session_id` is
passed) makes one extra LLM call to analyze feedback. Keep `CACHING=true` so session memory still works.
- **Pick a type without an LLM** when you need passages, not an answer:
`CHUNKS`, `CHUNKS_LEXICAL`, `SUMMARIES`, `CODE`, `SKILLS`. Completion types
make one LLM call; `GRAPH_COMPLETION_COT`, `_DECOMPOSITION`,
`_CONTEXT_EXTENSION`, `GRAPH_SUMMARY_COMPLETION` and `TEMPORAL` make
several; `NATURAL_LANGUAGE` makes one and retries (up to 3 attempts) only
on an empty or failed query; `FEELING_LUCKY` adds one to pick the type.
- **Narrow the search**: pass `datasets=[...]` (one search runs per dataset)
and a smaller `top_k` (per dataset, default 15; HYBRID caps each lane at
10, so only values below 10 shrink its context).
- `SESSION_SEARCH_MODE=concurrent` (default) overlaps the feedback analysis
with the answer; `sequential` runs them back to back.
### 7. Several datasets at once
With access control on (the default), each dataset has its own databases.
`DATASET_QUEUE_MAX_CONCURRENT` (default 6, from `DATABASE_MAX_LRU_CACHE_SIZE`)
caps how many datasets are processed at once in one process, and
`SUBPROCESS_IDLE_TTL_SECONDS` (600) keeps idle database workers warm.
`DATASET_QUEUE_ENABLED` (default true) enforces that cap, releases
subprocess engines when a dataset's last scope exits (they stay warm for
`SUBPROCESS_IDLE_TTL_SECONDS` and close at once only when it is 0) and pins
in-use engines against eviction.
Setting it false removes the cap rather than disabling parallelism, and
risks file-lock leaks and engine eviction under parallel load, so keep it
on.
### 8. Scaling a deployment
Open-source cognee scales **vertically, in one process**:
- The API server runs one worker (`gunicorn -w 1`), and the improve and
session locks, caches and semaphores are in-process only. More workers or
replicas against the same data are not coordinated (the Helm chart README
says to validate before scaling past one replica).
- The default embedded databases (SQLite, LanceDB, Ladybug) live on local
disk. For production, use Postgres + PGVector and a graph-native backend
such as Neo4j.
- `distributed/deploy/` holds one-click deploy templates (Modal, Fly,
Railway, Render, Daytona), not a distributed runner.
> **For production scale:** horizontal scaling (distributed ingestion across
> workers), cognee's enterprise GLiNER extraction, and the production
> Postgres graph adapter are part of cognee's proprietary offering. Contact
> social@cognee.ai.
## Pitfalls
- **`CHUNK_SIZE`, `CHUNK_OVERLAP`, `CHUNK_STRATEGY`** (in `.env.template`) and
`cognee.config.set_chunk_size()` etc. **have no effect on ingestion.** Pass
`chunk_size=` to `remember()` instead.
- **`LLM_RATE_LIMIT_TOKENS` and `EMBEDDING_RATE_LIMIT_TOKENS` do nothing**;
only request-count limits are enforced.
- **`FALLBACK_MODEL`** is used only when the main model rejects content on
policy grounds, not when it is slow, down, or rate-limited.
- **`data_per_batch=None` crashes.** Omit the argument to get the default.
- **A background `cognify()`** is not tracked by
`cognee.wait_for_background_tasks()` (a background `remember()` or
`improve()` is). Await its pipeline status yourself.
- **Big default chunks with a small local embedder.** The default chunk size
follows the configured embedding limit (8191), even for fastembed's
512-token model. Pass a smaller `chunk_size` for local embedding models.
- Benchmark with `CACHING=true`: with it off you measure cognee without its
memory layer.
## How it works
`remember()` → `add()` → `cognify()`: every document of the dataset is
scheduled at once, `data_per_batch` of them run concurrently, and each runs
the task chain (classify, chunk, extract graph + summarize, store). Tasks
receive their input in batches of `chunks_per_batch`.
`extract_graph_and_summarize` runs extraction and summarization together,
each gathering one LLM call per chunk in the batch. Every LLM call passes
through one process-wide rate limiter.
- Cognify task list and defaults: `cognee/api/v1/cognify/cognify.py`
- Concurrency (`data_per_batch` semaphore): `cognee/modules/pipelines/operations/run_tasks.py`
- Task batching: `cognee/modules/pipelines/tasks/task.py`, `run_tasks_base.py`
- LLM calls per chunk: `cognee/tasks/graph/extract_graph_and_summarize.py`
- Chunk-size default: `cognee/infrastructure/llm/utils.py:get_max_chunk_tokens`
- LLM config, rate limits, stage routing: `cognee/infrastructure/llm/config.py`,
`cognee/shared/rate_limiting.py`, `cognee/infrastructure/llm/overload_policy.py`,
`cognee/infrastructure/llm/retry_config.py`
- Embedding config: `cognee/infrastructure/databases/vector/embeddings/config.py`
- Dataset queue: `cognee/infrastructure/databases/dataset_queue/queue.py`
- Cost estimate: `cognee/modules/cognify/estimator.py`
## Extending it
- Benchmarks: `cognee/tests/performance/` (`batch_add_cognify_test.py`,
`locust_performance_analysis.py`, and `statistics_percentile/` for p50–p99
runs with mocked or replayed LLM calls). Measure before and after a change.
- Retrieval quality and cost: `cognee/eval_framework/` (`python -m
cognee.eval_framework`; Modal runs cost money).
- A new pipeline task that calls an LLM should keep `needs_llm=True` and go
through `LLMGateway`, so it shares the rate limiter and retry policy.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |