upstash/vector-js / skills
upstash-vector-js
VerifiedProvides quick-start guidance and a unified entry point for Vector features, SDK usage, and integrations. Use when users ask how to work with Vector, its TS SDK, features, or supported frameworks.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 1.4 KB |
| features/filtering-and-metadata.md | markdown | 2.1 KB |
| features/index-structure.md | markdown | 4.4 KB |
| features/namespaces.md | markdown | 929 B |
| sdk-methods.md | markdown | 3.5 KB |
features/index-structure.md
View on GitHub# Vector Feature: Index Structure
## Overview
Upstash Vector supports three major index structures:
- **Dense indexes**: semantic matching via dense embeddings.
- **Sparse indexes**: exact/near-exact token and feature matching.
- **Hybrid indexes**: combine dense + sparse results, optionally reranked.
---
## Key Concepts
### Sparse Vectors
- High‑dimensional, mostly‑zero representations.
- Represented as two equal‑length arrays:
- `indices`: int32 positions of non‑zero elements
- `values`: float32 values
- Upstash limit: **max 1,000 non‑zero entries per sparse vector**.
- Useful for exact token/word matching (BM25, SPLADE, etc.).
### Hybrid Vectors
- Combine dense semantic vectors with sparse exact‑match vectors.
- Hybrid queries run both dense + sparse search and fuse results.
- Require **both** components (`vector` and `sparseVector`).
### Fusion Algorithms
- **RRF (default)**: ranking‑based, simple, robust, ignores score magnitudes.
- **DBSF**: normalizes scores using distribution statistics; more sensitive to score ranges.
---
## Common Pitfalls
- **Hybrid upserts require both dense and sparse vectors**; omitting either fails.
- **Indexes with embedding models (dense, sparse or both)** let you upsert/query using text; indexes without embedding models do not.
- In Hybrin indexes, dense‑only or sparse‑only querying is allowed, but **fusion happens only when both are provided**.
---
## Usage with Embedding Models
```ts
await index.upsert([
{
id: "t1",
data: "Upstash Vector provides sparse models.",
},
]);
const results = await index.query({
data: "Upstash Vector",
topK: 5,
});
import { WeightingStrategy } from "@upstash/vector";
await index.query({
data: "Upstash Vector",
weightingStrategy: WeightingStrategy.IDF,
});
```
## Sparse Index Usage
### Upserting Sparse Vectors
```ts
await index.upsert([
{
id: "x1",
sparseVector: {
indices: [1, 2, 3],
values: [0.1, 0.2, 0.3],
},
},
]);
const results = await index.query({
sparseVector: {
indices: [3, 5],
values: [0.3, 0.5],
},
topK: 5,
includeMetadata: true,
});
```
- Scores use **inner product**, matching only overlapping indices.
- Results may be fewer than `top_k` if no overlapping dims exist.
## Hybrid Index Usage
### Upserting Dense + Sparse
```ts
await index.upsert([
{
id: "h1",
vector: [0.1, 0.5],
sparseVector: {
indices: [1, 2],
values: [0.1, 0.2],
},
},
]);
const results = await index.query({
vector: [0.5, 0.4],
sparseVector: {
indices: [3, 5],
values: [0.3, 0.5],
},
topK: 5,
});
import { FusionAlgorithm } from "@upstash/vector";
await index.query({
vector: [0.5, 0.4],
sparseVector: {
indices: [2, 3],
values: [0.1, 0.2],
},
fusionAlgorithm: FusionAlgorithm.RRF, // or FusionAlgorithm.DBSF
});
```
---
## Custom Reranking
Sometimes RRF/DBSF is insufficient (e.g., using bge‑reranker-v2-m3). Query dense and sparse portions separately and rerank in your own model.
### Custom Rerank (vector input)
```ts
// Dense-only
const dense = await index.query({
vector: [0.5, 0.4],
topK: 5,
});
// Sparse-only
const sparse = await index.query({
sparseVector: {
indices: [3, 5],
values: [0.3, 0.5],
},
topK: 5,
});
// Custom rerank dense + sparse...
```
### Custom Rerank (text input with hosted models)
```ts
import { QueryMode } from "@upstash/vector";
const dense = await index.query({
data: "Upstash Vector",
queryMode: QueryMode.DENSE,
});
const sparse = await index.query({
data: "Upstash Vector",
queryMode: QueryMode.SPARSE,
});
// Rerank...
```
---
## Agent Implementation Notes
- Always check whether the index supports **hosted embeddings** before using `data=` fields.
- Use `await index.info()` to check: if `denseIndex?.embeddingModel` or `sparseIndex?.embeddingModel` exists, the index supports text-based embeddings via the `data` field.
- Example:
```ts
const info = await index.info();
const hasDenseEmbedding = !!info.denseIndex?.embeddingModel;
const hasSparseEmbedding = !!info.sparseIndex?.embeddingModel;
```
- For hybrid indexes:
- Always provide _both_ `vector` and `sparseVector` unless intentionally querying only one modality.
- For reranking workflows:
- Use dense-only + sparse-only queries, never hybrid queries (as fusion is already applied).
- When building sparse vectors manually:
- Ensure indices are sorted, unique, and below the model dimension.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |