a5c-ai/babysitter / library/specializations/data-engineering-analytics/skills/kafka-topic-designer
Kafka Topic Designer
VerifiedDesigns and optimizes Apache Kafka topics and configurations
SKILL.md
View on GitHub---
name: Kafka Topic Designer
description: Designs and optimizes Apache Kafka topics and configurations
version: 1.0.0
category: Streaming
skillId: SK-DEA-007
allowed-tools:
- Read
- Write
- Edit
- Glob
- Grep
- Bash
graph:
domains: [domain:data-engineering]
specializations: [specialization:data-engineering-analytics]
skillAreas: [skill-area:messaging-queuing, skill-area:streaming-realtime-processing]
roles: [role:data-engineer, role:analytics-engineer]
workflows: [workflow:data-pipeline-deployment]
topics: [topic:event-sourcing, topic:event-driven-architecture]
---
# Kafka Topic Designer
## Overview
Designs and optimizes Apache Kafka topics and configurations. This skill provides expertise in topic architecture, partitioning strategies, and producer/consumer configuration for optimal streaming performance.
## Capabilities
- Topic naming convention design
- Partition strategy optimization
- Replication factor recommendations
- Retention policy configuration
- Compaction strategy design
- Schema registry integration
- Consumer group design
- Throughput capacity planning
- Security configuration (ACLs, encryption)
## Input Schema
```json
{
"requirements": {
"throughputMBps": "number",
"messageSize": "number",
"retentionDays": "number",
"orderingRequirements": "string"
},
"existingTopics": ["object"],
"clusterConfig": "object"
}
```
## Output Schema
```json
{
"topicDesign": {
"name": "string",
"partitions": "number",
"replicationFactor": "number",
"configs": "object"
},
"schemaDefinition": "object",
"producerConfig": "object",
"consumerConfig": "object"
}
```
## Target Processes
- Streaming Pipeline
- ETL/ELT Pipeline (CDC)
- Feature Store Setup
## Usage Guidelines
1. Define throughput and latency requirements
2. Specify message size and retention needs
3. Document ordering requirements (per-partition, global)
4. Include existing topic configurations for consistency
## Best Practices
- Use consistent naming conventions across all topics
- Size partitions based on consumer parallelism needs
- Configure appropriate retention for compliance and replay scenarios
- Implement schema registry for schema evolution
- Set up proper ACLs for security
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |