Skip to content

lidge-jun/codexclaw / plugins/codexclaw/skills/dev-devops

cxc-dev-devops

Verified

Use for infra, CI/CD, releases, repo policy, worktree cleanup and native desktop apps. Triggers: Dockerfile, K8s, IaC, SRE, Tauri, 스택 PR CI, 배포, 인프라, 쿠버네티스, 브랜치 정리, 브랜치 삭제, 워크트리 정리, 저장소 세팅, 브랜치 보호, 에이전트 PR, PR 정책, 데스크톱 앱, 메뉴 막대.

references/ml-infra.md

View on GitHub
# ML Infrastructure — GPU Clusters, Model Registry, Serving

Last reviewed: 2026-06-16
Applies to: NVIDIA GPU clusters, model serving platforms, MLOps
When to read: ML infrastructure provisioning or model deployment
Canonical owner: dev-devops (infra layer); dev-backend owns API/serving code patterns

---

## §1 GPU Cluster Management

### Node Selection

| GPU | VRAM | Best For | K8s Resource |
|-----|------|----------|-------------|
| A100 (40/80GB) | 40-80GB | Training, large model inference | `nvidia.com/gpu: 1` |
| H100 (80GB) | 80GB | Large-scale training, HPC | `nvidia.com/gpu: 1` |
| L4 (24GB) | 24GB | Inference, cost-efficient | `nvidia.com/gpu: 1` |
| T4 (16GB) | 16GB | Light inference, dev/test | `nvidia.com/gpu: 1` |

### K8s GPU Scheduling

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: model-server
spec:
  template:
    spec:
      containers:
        - name: server
          image: model-server@sha256:abc123
          resources:
            limits:
              nvidia.com/gpu: 1
              memory: "32Gi"
            requests:
              nvidia.com/gpu: 1
              memory: "16Gi"
      nodeSelector:
        accelerator: nvidia-l4
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
```

### Cost Optimization

| Strategy | Detail |
|----------|--------|
| Spot/preemptible | Training workloads (checkpointed) |
| Right-sizing | Match GPU VRAM to model size |
| Time-sharing | NVIDIA MPS for small inference models |
| Autoscaling | Scale-to-zero with KEDA for inference |

---

## §2 Model Registry

### Registry Selection

| Tool | Best For | Key Feature |
|------|----------|-------------|
| MLflow Model Registry | General MLOps | Versioning, staging, lineage |
| Weights & Biases | Experiment tracking + registry | Artifacts, sweeps, reports |
| HuggingFace Hub | Open-source models, LoRA adapters | Model cards, datasets |
| SageMaker Model Registry | AWS-native MLOps | Approval workflows |

### Model Versioning Contract

```
models/
  payments-fraud-detector/
    v1.0.0/
      model.onnx
      config.json
      metrics.json     # eval metrics at registration
      requirements.txt # inference dependencies
    v1.1.0/
      ...
```

| Rule | Detail |
|------|--------|
| SemVer | Major = breaking API change, Minor = improved metrics, Patch = fix |
| Immutable | Once registered, never overwrite — create new version |
| Metrics | Eval metrics recorded at registration time |
| Lineage | Link to training run, dataset version, code commit |

---

## §3 Model Serving Patterns

### Serving Architecture Decision

| Pattern | When | Tool |
|---------|------|------|
| REST/gRPC API | Low-medium throughput, general models | FastAPI, TorchServe, Triton |
| Batch inference | Large dataset, latency-tolerant | Spark, Ray, SageMaker Batch |
| Streaming | Real-time features, event-driven | Kafka + model sidecar |
| Edge inference | Latency-critical, privacy-sensitive | ONNX Runtime, TF Lite, Core ML |

### K8s Serving with Scale-to-Zero

```yaml
# KEDA ScaledObject for inference service
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: model-server-scaler
spec:
  scaleTargetRef:
    name: model-server
  minReplicaCount: 0  # scale to zero when idle
  maxReplicaCount: 10
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus:9090
        metricName: http_requests_total
        query: rate(http_requests_total{service="model-server"}[2m])
        threshold: "10"
  cooldownPeriod: 300  # 5min before scale-down
```

### Health Checks for Model Servers

```python
# FastAPI model server health
@app.get("/health")
async def health():
    return {"status": "ok", "model_loaded": model is not None}

@app.get("/ready")
async def ready():
    # Verify model can actually inference
    try:
        _ = model.predict(dummy_input)
        return {"status": "ready"}
    except Exception as e:
        return JSONResponse({"status": "not_ready", "error": str(e)}, status_code=503)
```

---

## §4 Edge Inference Triage

### Decision: Cloud vs Edge Inference

| Factor | Cloud | Edge |
|--------|-------|------|
| Latency requirement | >100ms acceptable | <50ms required |
| Model size | >1GB | <500MB (quantized) |
| Privacy | Data can leave device | Data must stay on device |
| Connectivity | Reliable network | Intermittent/offline |
| Update frequency | Frequent model updates | Infrequent, versioned |

### Edge Model Optimization Pipeline

```
Full model (PyTorch/TF)
  → Export to ONNX
  → Quantize (INT8/FP16)
  → Optimize (ONNX Runtime, TensorRT, Core ML)
  → Package with inference runtime
  → Deploy to edge device
```

### Runtime Selection

| Runtime | Platform | Best For |
|---------|----------|----------|
| ONNX Runtime | Cross-platform | General inference, web |
| TensorRT | NVIDIA GPU | GPU-optimized inference |
| Core ML | Apple devices | iOS/macOS on-device |
| TF Lite | Mobile/embedded | Android, IoT |

---

## §5 MLOps Platform Patterns

### Training Pipeline

```
Data versioning (DVC/LakeFS)
  → Feature engineering (dbt/Spark)
  → Training (distributed, GPU cluster)
  → Evaluation (automated metrics + human review)
  → Registration (model registry)
  → Deployment (CI/CD → serving infra)
  → Monitoring (data drift, model performance)
```

### Monitoring

| Signal | What to Watch | Alert On |
|--------|---------------|----------|
| Data drift | Feature distribution shift | KL divergence > threshold |
| Model performance | Accuracy, F1, latency | Degradation from baseline |
| Prediction distribution | Output class balance | Unexpected shift |
| Infrastructure | GPU utilization, memory, throughput | Saturation > 90% |

---

## §6 Anti-Patterns

| Banned | Symptom | Fix |
|--------|---------|-----|
| GPU always-on for batch | Burning money when idle | Scale-to-zero with KEDA |
| No model versioning | Can't rollback, can't reproduce | Model registry with SemVer |
| Training on spot without checkpoints | Lost hours of compute | Checkpoint every N steps |
| Serving without health checks | Silent model loading failures | `/health` + `/ready` endpoints |
| No data drift monitoring | Silent accuracy degradation | Automated drift detection |
| One-size GPU | Overpaying for inference | Right-size: L4/T4 for inference, A100/H100 for training |
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 319kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall