Skip to content

a5c-ai/babysitter / library/specializations/data-engineering-analytics/skills/kafka-topic-designer

Kafka Topic Designer

Verified

Designs and optimizes Apache Kafka topics and configurations

Kafka Topic Designer

Overview

Designs and optimizes Apache Kafka topics and configurations. This skill provides expertise in topic architecture, partitioning strategies, and producer/consumer configuration for optimal streaming performance.

Capabilities

  • Topic naming convention design
  • Partition strategy optimization
  • Replication factor recommendations
  • Retention policy configuration
  • Compaction strategy design
  • Schema registry integration
  • Consumer group design
  • Throughput capacity planning
  • Security configuration (ACLs, encryption)

Input Schema

{
  "requirements": {
    "throughputMBps": "number",
    "messageSize": "number",
    "retentionDays": "number",
    "orderingRequirements": "string"
  },
  "existingTopics": ["object"],
  "clusterConfig": "object"
}

Output Schema

{
  "topicDesign": {
    "name": "string",
    "partitions": "number",
    "replicationFactor": "number",
    "configs": "object"
  },
  "schemaDefinition": "object",
  "producerConfig": "object",
  "consumerConfig": "object"
}

Target Processes

  • Streaming Pipeline
  • ETL/ELT Pipeline (CDC)
  • Feature Store Setup

Usage Guidelines

  1. Define throughput and latency requirements
  2. Specify message size and retention needs
  3. Document ordering requirements (per-partition, global)
  4. Include existing topic configurations for consistency

Best Practices

  • Use consistent naming conventions across all topics
  • Size partitions based on consumer parallelism needs
  • Configure appropriate retention for compliance and replay scenarios
  • Implement schema registry for schema evolution
  • Set up proper ACLs for security
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall