Skip to content

giuseppe-trisciuoglio/developer-kit / plugins/developer-kit-ai/skills/rag

Implements document chunking, embedding generation, vector storage, and retrieval pipelines for Retrieval-Augmented Generation systems. Use when building RAG applications, creating document Q&A systems, or integrating AI with knowledge bases.

references/vector-databases.md

View on GitHub
# Vector Database Comparison and Configuration

## Overview
Vector databases store and efficiently retrieve document embeddings for semantic search in RAG systems.

## Popular Vector Database Options

### 1. Pinecone
- **Type**: Managed cloud service
- **Features**: Scalable, fast queries, managed infrastructure
- **Use Case**: Production applications requiring high availability

### 2. Weaviate
- **Type**: Open-source, hybrid search
- **Features**: Combines vector and keyword search, GraphQL API
- **Use Case**: Applications needing both semantic and traditional search

### 3. Milvus
- **Type**: High performance, on-premise
- **Features**: Distributed architecture, GPU acceleration
- **Use Case**: Large-scale deployments with custom infrastructure

### 4. Chroma
- **Type**: Lightweight, easy to use
- **Features**: Local deployment, simple API
- **Use Case**: Development and small-scale applications

### 5. Qdrant
- **Type**: Fast, filtered search
- **Features**: Advanced filtering, payload support
- **Use Case**: Applications requiring complex metadata filtering

### 6. FAISS
- **Type**: Meta's library, local deployment
- **Features**: High performance, CPU/GPU optimized
- **Use Case**: Research and applications needing full control

## Configuration Examples

### Pinecone Setup
```python
import pinecone
from langchain.vectorstores import Pinecone

pinecone.init(api_key="your-api-key", environment="us-west1-gcp")
index = pinecone.Index("your-index-name")
vectorstore = Pinecone(index, embeddings.embed_query, "text")
```

### Weaviate Setup
```python
import weaviate
from langchain.vectorstores import Weaviate

client = weaviate.Client("http://localhost:8080")
vectorstore = Weaviate(client, "Document", "content", embeddings)
```

### Chroma Local Setup
```python
from langchain.vectorstores import Chroma

vectorstore = Chroma(
    collection_name="my_collection",
    embedding_function=embeddings,
    persist_directory="./chroma_db"
)
```

## Selection Criteria

1. **Scale**: Number of documents and expected query volume
2. **Performance**: Latency requirements and throughput needs
3. **Deployment**: Cloud vs on-premise preferences
4. **Features**: Filtering, hybrid search, metadata support
5. **Cost**: Budget constraints and operational overhead
6. **Maintenance**: Team expertise and available resources

## Best Practices

1. **Indexing Strategy**: Choose appropriate distance metrics (cosine, euclidean)
2. **Sharding**: Distribute data for large-scale deployments
3. **Monitoring**: Track query performance and system health
4. **Backups**: Implement regular backup procedures
5. **Security**: Secure access to sensitive data
6. **Optimization**: Tune parameters for your specific use case
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 321kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall