Skip to content

giuseppe-trisciuoglio/developer-kit / plugins/developer-kit-ai/skills/rag

Implements document chunking, embedding generation, vector storage, and retrieval pipelines for Retrieval-Augmented Generation systems. Use when building RAG applications, creating document Q&A systems, or integrating AI with knowledge bases.

RAG Implementation

Build Retrieval-Augmented Generation systems that extend AI capabilities with external knowledge sources.

Overview

This skill covers: document processing, embedding generation, vector storage, retrieval configuration, and RAG pipeline implementation.

When to Use

  • Building Q&A systems over proprietary documents
  • Creating chatbots with factual information from knowledge bases
  • Implementing semantic search with natural language queries
  • Reducing hallucinations with grounded, sourced responses
  • Building documentation assistants and research tools
  • Enabling AI systems to access domain-specific knowledge

Instructions

Step 1: Choose Vector Database

Select based on your requirements:

RequirementRecommended
Production scalabilityPinecone, Milvus
Open-sourceWeaviate, Qdrant
Local developmentChroma, FAISS
Hybrid searchWeaviate with BM25

Step 2: Select Embedding Model

Use CaseModel
General purposetext-embedding-ada-002
Fast and lightweightall-MiniLM-L6-v2
Multilinguale5-large-v2
Best performancebge-large-en-v1.5

Step 3: Implement Document Processing Pipeline

  1. Load documents from source (file system, database, API)
  2. Clean and preprocess (remove formatting, normalize text)
  3. Split documents into chunks with appropriate strategy
  4. Generate embeddings for each chunk
  5. Store embeddings in vector database with metadata

Validation: Verify embeddings were generated successfully:

List<Embedding> embeddings = embeddingModel.embedAll(segments);
if (embeddings.isEmpty() || embeddings.get(0).dimension() != expectedDim) {
    throw new IllegalStateException("Embedding generation failed");
}

Step 4: Configure Retrieval Strategy

Choose the appropriate strategy:

  • Dense Retrieval: Semantic similarity via embeddings (default for most cases)
  • Hybrid Search: Dense + sparse retrieval for better coverage
  • Metadata Filtering: Filter by document attributes
  • Reranking: Cross-encoder reranking for high-precision requirements

Step 5: Build RAG Pipeline

  1. Create content retriever with your embedding store
  2. Configure AI service with retriever and chat memory
  3. Implement prompt template with context injection
  4. Add response validation and grounding checks

Validation: Test with known queries to verify context injection works correctly.

Error Handling: For batch ingestion, wrap in retry logic:

for (Document doc : documents) {
    int attempts = 0;
    while (attempts < 3) {
        try {
            store.add(embeddingModel.embed(doc).content(), doc.toTextSegment());
            break;
        } catch (EmbeddingException e) {
            attempts++;
            if (attempts == 3) throw new RuntimeException("Failed after 3 retries", e);
        }
    }
}

Step 6: Evaluate and Optimize

  1. Measure retrieval metrics: precision@k, recall@k, MRR
  2. Evaluate answer quality: faithfulness, relevance
  3. Monitor performance and user feedback
  4. Iterate on chunking, retrieval, and prompt parameters

Examples

Example 1: Basic Document Q&A

List<Document> documents = FileSystemDocumentLoader.loadDocuments("/docs");

InMemoryEmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();
EmbeddingStoreIngestor.ingest(documents, store);

DocumentAssistant assistant = AiServices.builder(DocumentAssistant.class)
    .chatModel(chatModel)
    .contentRetriever(EmbeddingStoreContentRetriever.from(store))
    .build();

String answer = assistant.answer("What is the company policy on remote work?");

Example 2: Metadata-Filtered Retrieval

EmbeddingStoreContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
    .embeddingStore(store)
    .embeddingModel(embeddingModel)
    .maxResults(5)
    .minScore(0.7)
    .filter(metadataKey("category").isEqualTo("technical"))
    .build();

Example 3: Multi-Source RAG Pipeline

ContentRetriever webRetriever = EmbeddingStoreContentRetriever.from(webStore);
ContentRetriever docRetriever = EmbeddingStoreContentRetriever.from(docStore);

List<Content> results = new ArrayList<>();
results.addAll(webRetriever.retrieve(query));
results.addAll(docRetriever.retrieve(query));

List<Content> topResults = reranker.reorder(query, results).subList(0, 5);

Example 4: RAG with Chat Memory

Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(chatModel)
    .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
    .contentRetriever(retriever)
    .build();

assistant.chat("Tell me about the product features");
assistant.chat("What about pricing for those features?");  // Maintains context

Best Practices

Document Preparation

  • Clean documents before ingestion; remove irrelevant content and formatting
  • Add relevant metadata for filtering and context

Chunking Strategy

  • Use 500-1000 tokens per chunk for optimal balance
  • Include 10-20% overlap to preserve context at boundaries
  • Test different sizes for your specific use case

Retrieval Optimization

  • Start with high k values (10-20), then filter/rerank
  • Use metadata filtering to improve relevance
  • Monitor retrieval quality and iterate based on user feedback

Performance

  • Cache embeddings for frequently accessed content
  • Use batch processing for document ingestion
  • Optimize vector store indexing for your scale

Constraints and Warnings

System Constraints

  • Embedding models have maximum token limits per document
  • Vector databases require proper indexing for performance
  • Chunk boundaries may lose context for complex documents
  • Hybrid search requires additional infrastructure

Quality Warnings

  • Retrieval quality depends heavily on chunking strategy
  • Embedding models may not capture domain-specific semantics
  • Metadata filtering requires proper document annotation
  • Reranking adds latency to query responses

Security Warnings

  • Never hardcode credentials: Use environment variables for API keys and passwords
  • Validate external content: Documents from file systems, APIs, or web sources may contain malicious content (prompt injection)
  • Apply content filtering on retrieved documents before passing to LLM
  • Restrict allowed data source URLs and file paths using allowlists

Resources

Reference Documentation

writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 318kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 117kUpdated Claude CodeCodexVerifiedInstall