Skip to content

upstash/vector-js / skills

upstash-vector-js

Verified

Provides quick-start guidance and a unified entry point for Vector features, SDK usage, and integrations. Use when users ask how to work with Vector, its TS SDK, features, or supported frameworks.

FileKindSize
SKILL.mdskill md1.4 KB
features/filtering-and-metadata.mdmarkdown2.1 KB
features/index-structure.mdmarkdown4.4 KB
features/namespaces.mdmarkdown929 B
sdk-methods.mdmarkdown3.5 KB

features/index-structure.md

View on GitHub
# Vector Feature: Index Structure

## Overview

Upstash Vector supports three major index structures:

- **Dense indexes**: semantic matching via dense embeddings.
- **Sparse indexes**: exact/near-exact token and feature matching.
- **Hybrid indexes**: combine dense + sparse results, optionally reranked.

---

## Key Concepts

### Sparse Vectors

- High‑dimensional, mostly‑zero representations.
- Represented as two equal‑length arrays:
  - `indices`: int32 positions of non‑zero elements
  - `values`: float32 values
- Upstash limit: **max 1,000 non‑zero entries per sparse vector**.
- Useful for exact token/word matching (BM25, SPLADE, etc.).

### Hybrid Vectors

- Combine dense semantic vectors with sparse exact‑match vectors.
- Hybrid queries run both dense + sparse search and fuse results.
- Require **both** components (`vector` and `sparseVector`).

### Fusion Algorithms

- **RRF (default)**: ranking‑based, simple, robust, ignores score magnitudes.
- **DBSF**: normalizes scores using distribution statistics; more sensitive to score ranges.

---

## Common Pitfalls

- **Hybrid upserts require both dense and sparse vectors**; omitting either fails.
- **Indexes with embedding models (dense, sparse or both)** let you upsert/query using text; indexes without embedding models do not.
- In Hybrin indexes, dense‑only or sparse‑only querying is allowed, but **fusion happens only when both are provided**.

---

## Usage with Embedding Models

```ts
await index.upsert([
  {
    id: "t1",
    data: "Upstash Vector provides sparse models.",
  },
]);

const results = await index.query({
  data: "Upstash Vector",
  topK: 5,
});

import { WeightingStrategy } from "@upstash/vector";
await index.query({
  data: "Upstash Vector",
  weightingStrategy: WeightingStrategy.IDF,
});
```

## Sparse Index Usage

### Upserting Sparse Vectors

```ts
await index.upsert([
  {
    id: "x1",
    sparseVector: {
      indices: [1, 2, 3],
      values: [0.1, 0.2, 0.3],
    },
  },
]);

const results = await index.query({
  sparseVector: {
    indices: [3, 5],
    values: [0.3, 0.5],
  },
  topK: 5,
  includeMetadata: true,
});
```

- Scores use **inner product**, matching only overlapping indices.
- Results may be fewer than `top_k` if no overlapping dims exist.

## Hybrid Index Usage

### Upserting Dense + Sparse

```ts
await index.upsert([
  {
    id: "h1",
    vector: [0.1, 0.5],
    sparseVector: {
      indices: [1, 2],
      values: [0.1, 0.2],
    },
  },
]);

const results = await index.query({
  vector: [0.5, 0.4],
  sparseVector: {
    indices: [3, 5],
    values: [0.3, 0.5],
  },
  topK: 5,
});

import { FusionAlgorithm } from "@upstash/vector";
await index.query({
  vector: [0.5, 0.4],
  sparseVector: {
    indices: [2, 3],
    values: [0.1, 0.2],
  },
  fusionAlgorithm: FusionAlgorithm.RRF, // or FusionAlgorithm.DBSF
});
```

---

## Custom Reranking

Sometimes RRF/DBSF is insufficient (e.g., using bge‑reranker-v2-m3). Query dense and sparse portions separately and rerank in your own model.

### Custom Rerank (vector input)

```ts
// Dense-only
const dense = await index.query({
  vector: [0.5, 0.4],
  topK: 5,
});

// Sparse-only
const sparse = await index.query({
  sparseVector: {
    indices: [3, 5],
    values: [0.3, 0.5],
  },
  topK: 5,
});

// Custom rerank dense + sparse...
```

### Custom Rerank (text input with hosted models)

```ts
import { QueryMode } from "@upstash/vector";

const dense = await index.query({
  data: "Upstash Vector",
  queryMode: QueryMode.DENSE,
});

const sparse = await index.query({
  data: "Upstash Vector",
  queryMode: QueryMode.SPARSE,
});

// Rerank...
```

---

## Agent Implementation Notes

- Always check whether the index supports **hosted embeddings** before using `data=` fields.
  - Use `await index.info()` to check: if `denseIndex?.embeddingModel` or `sparseIndex?.embeddingModel` exists, the index supports text-based embeddings via the `data` field.
  - Example:
    ```ts
    const info = await index.info();
    const hasDenseEmbedding = !!info.denseIndex?.embeddingModel;
    const hasSparseEmbedding = !!info.sparseIndex?.embeddingModel;
    ```
- For hybrid indexes:
  - Always provide _both_ `vector` and `sparseVector` unless intentionally querying only one modality.
- For reranking workflows:
  - Use dense-only + sparse-only queries, never hybrid queries (as fusion is already applied).
- When building sparse vectors manually:
  - Ensure indices are sorted, unique, and below the model dimension.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall