a5c-ai/babysitter / library/specializations/data-engineering-analytics/skills/kafka-topic-designer
Kafka Topic Designer
VerifiedDesigns and optimizes Apache Kafka topics and configurations
Kafka Topic Designer
Overview
Designs and optimizes Apache Kafka topics and configurations. This skill provides expertise in topic architecture, partitioning strategies, and producer/consumer configuration for optimal streaming performance.
Capabilities
- Topic naming convention design
- Partition strategy optimization
- Replication factor recommendations
- Retention policy configuration
- Compaction strategy design
- Schema registry integration
- Consumer group design
- Throughput capacity planning
- Security configuration (ACLs, encryption)
Input Schema
{
"requirements": {
"throughputMBps": "number",
"messageSize": "number",
"retentionDays": "number",
"orderingRequirements": "string"
},
"existingTopics": ["object"],
"clusterConfig": "object"
}
Output Schema
{
"topicDesign": {
"name": "string",
"partitions": "number",
"replicationFactor": "number",
"configs": "object"
},
"schemaDefinition": "object",
"producerConfig": "object",
"consumerConfig": "object"
}
Target Processes
- Streaming Pipeline
- ETL/ELT Pipeline (CDC)
- Feature Store Setup
Usage Guidelines
- Define throughput and latency requirements
- Specify message size and retention needs
- Document ordering requirements (per-partition, global)
- Include existing topic configurations for consistency
Best Practices
- Use consistent naming conventions across all topics
- Size partitions based on consumer parallelism needs
- Configure appropriate retention for compliance and replay scenarios
- Implement schema registry for schema evolution
- Set up proper ACLs for security
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |