metalbear-co/mirrord / skills/skills/mirrord-kafka
mirrord-kafka
VerifiedHelps DevOps engineers configure mirrord Operator's Kafka queue splitting feature end-to-end. Generates the MirrordSplitConfig and MirrordPropertyList Kubernetes CRD YAMLs (the current resources; MirrordKafkaTopicsConsumer + MirrordKafkaClientConfig are deprecated but still supported), the matching mirrord.json split_queues section with message_filter and jq_filter, and Helm value guidance. Use this skill whenever the user mentions Kafka splitting with mirrord, MirrordSplitConfig, MirrordPropertyList, MirrordKafkaClientConfig, MirrordKafkaTopicsConsumer, Kafka queue/topic splitting, configuring mirrord with Kafka, Kafka Streams splitting, MSK IAM auth, or troubleshooting Kafka splitting sessions. Also trigger on split_queues with queue_type Kafka, or connecting mirrord to a Kafka cluster. This is a Team/Enterprise feature of mirrord.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 25.8 KB |
| references/kafka-client-config-crd.md | markdown | 5.5 KB |
| references/kafka-topics-consumer-crd.md | markdown | 4.2 KB |
| references/known-issues.md | markdown | 11.0 KB |
| references/mirrord-property-list-crd.md | markdown | 16.9 KB |
| references/mirrord-split-config-crd.md | markdown | 15.9 KB |
references/kafka-client-config-crd.md
View on GitHub# MirrordKafkaClientConfig CRD Reference (DEPRECATED)
> **⚠️ Deprecated.** `MirrordKafkaClientConfig` is replaced by **`MirrordPropertyList`** (`references/mirrord-property-list-crd.md`). Existing resources still work, and `MirrordSplitConfig`'s `clientConfig` falls back to a `MirrordKafkaClientConfig` of the same name in the operator namespace. Use this file only when helping maintain or migrate a legacy setup; generate `MirrordPropertyList` for anything new.
## Overview
Configures the mirrord Operator's Kafka client for queue splitting. Must be created in the **operator's namespace** (typically `mirrord`).
**API Version:** `queues.mirrord.metalbear.co/v1alpha`
**Kind:** `MirrordKafkaClientConfig`
## spec fields
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `properties` | list of `{name, value}` | Yes | Kafka client properties passed as a `.properties` file. Full list: [librdkafka CONFIGURATION.md](https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md) |
| `parent` | string | No | Name of another `MirrordKafkaClientConfig` in the same namespace to inherit from. Child properties override parent; setting `value: null` removes an inherited property. |
| `loadFromSecret` | string | No | Reference to a K8s Secret in `<namespace>/<name>` format. Each key-value in the secret becomes a property. |
| `authenticationExtra` | object | No | Custom auth methods that go beyond client properties. |
### properties[] entry
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | Yes | Kafka client property name (e.g. `bootstrap.servers`) |
| `value` | string or null | Yes | Property value. Use `null` only when inheriting from parent to remove a property. |
### authenticationExtra
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `kind` | string | Yes | Authentication type. Currently supported: `MSK_IAM` |
| `awsRegion` | string | Yes (for MSK_IAM) | AWS region for MSK IAM auth |
When `kind: MSK_IAM`, two properties are auto-merged:
- `sasl.mechanism=OAUTHBEARER`
- `security.protocol=SASL_SSL`
## Property resolution priority (highest to lowest)
1. Child `spec.properties`
2. Child `spec.loadFromSecret`
3. Parent `spec.properties`
4. Parent `spec.loadFromSecret`
## Important constraints
- **Namespace:** Must always be in the operator's namespace (default: `mirrord`)
- **group.id:** Always overwritten by the operator at runtime — do not set it
- **Secret access:** By default the operator only has read access to secrets in the operator's namespace
## Common properties
| Property | Typical value | Notes |
|----------|---------------|-------|
| `bootstrap.servers` | `kafka.default.svc.cluster.local:9092` | Required for any setup |
| `client.id` | `mirrord-operator` | Optional identifier |
| `security.protocol` | `SASL_SSL`, `SSL`, `PLAINTEXT` | Depends on cluster auth |
| `sasl.mechanism` | `PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512`, `OAUTHBEARER` | For SASL auth |
| `sasl.username` / `sasl.password` | credentials | For SASL PLAIN/SCRAM |
| `ssl.certificate.pem` | PEM content | Client cert for mTLS |
| `ssl.key.pem` | PEM content | Client private key |
| `ssl.ca.pem` | PEM content | CA cert to verify broker |
| `ssl.key.password` | string | If private key is password-protected |
## Authentication patterns
### Plain/no auth (dev clusters)
```yaml crd
spec:
properties:
- name: bootstrap.servers
value: kafka.default.svc.cluster.local:9092
```
### SASL PLAIN
> **Security:** Never inline `sasl.password` (or any credential) as a literal value in generated YAML. Reference a Kubernetes Secret in the operator's namespace via `loadFromSecret`. The user creates the Secret themselves; the agent must not handle the password value.
```yaml crd
spec:
# Secret in the operator namespace with keys: sasl.username, sasl.password
loadFromSecret: mirrord/kafka-sasl-creds
properties:
- name: bootstrap.servers
value: kafka-broker:9093
- name: security.protocol
value: SASL_SSL
- name: sasl.mechanism
value: PLAIN
```
The user creates the backing Secret from files, not literals (so credentials don't appear in shell history):
```sh
# User saves username/password to files, then:
kubectl create secret generic kafka-sasl-creds \
--from-file=sasl.username=/path/to/username-file \
--from-file=sasl.password=/path/to/password-file \
-n mirrord
# User: delete the temporary files after the Secret is created.
```
### SSL/mTLS (via secret)
```yaml crd
spec:
loadFromSecret: mirrord/kafka-ssl-creds
properties:
- name: bootstrap.servers
value: kafka-broker:9093
- name: security.protocol
value: SSL
```
### MSK IAM (AWS)
```yaml crd
spec:
authenticationExtra:
kind: MSK_IAM
awsRegion: us-east-1
properties:
- name: bootstrap.servers
value: b-1.mycluster.kafka.us-east-1.amazonaws.com:9098
```
### Inheritance example
```yaml crd
# Parent
apiVersion: queues.mirrord.metalbear.co/v1alpha
kind: MirrordKafkaClientConfig
metadata:
name: base-config
namespace: mirrord
spec:
properties:
- name: bootstrap.servers
value: kafka.default.svc.cluster.local:9092
- name: message.send.max.retries
value: "4"
---
# Child (inherits + overrides)
apiVersion: queues.mirrord.metalbear.co/v1alpha
kind: MirrordKafkaClientConfig
metadata:
name: with-auth
namespace: mirrord
spec:
parent: base-config
properties:
- name: security.protocol
value: SASL_SSL
- name: message.send.max.retries
value: null # removes inherited property
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |