anyshift-io/sre-skills / skills/kubectl-investigator
kubectl-investigator
UnverifiedInvestigate a live or recent incident in a Kubernetes cluster. Anchor the window, bisect the change surface (rollouts, ConfigMaps/Secrets, RBAC, HPA/cluster changes, CronJobs), classify against four reference failure paths (OOM, DNS, cascading-failure, deploy-correlator), confirm the hypothesis with three independent signals, quantify blast radius, and propose mitigation before root cause. Use whenever an agent is asked "what is breaking in the cluster right now", "why did this pod/Deployment just page", "did the rollout cause Z", or to triage an active Kubernetes incident. Vendor-neutral by default (works with kubectl, kube-state-metrics, and whatever telemetry you have); an opt-in Anyshift integration is documented separately.
tests/eval/run_until_pass.sh
View on GitHub#!/usr/bin/env bash
# Run the ablation eval fixture-by-fixture, retrying each fixture until BOTH arms
# (control + treatment) have all TRIALS trials. Leans on run_eval.py's resume
# (fills only missing trials) and per-call backoff. Safe to interrupt and re-run.
#
# Usage: bash tests/eval/run_until_pass.sh [TRIALS] [MAX_ATTEMPTS_PER_FIXTURE]
set -u
cd "$(dirname "$0")/../.." # -> skills/kubectl-investigator
TRIALS="${1:-1}"
MAX_ATTEMPTS="${2:-12}"
OUT="tests/eval/eval_results.json"
FIXTURES=(
01-oom-cascade
02-dns-resolution-failure
03-cascading-failure-retry-storm
04-deploy-correlator-serialization
05-outside-reference-paths-third-party-rate-limit
06-ambiguous-t0-slow-burn
07-blast-radius-asymmetric-revert
08-deploy-correlator-confirmation-bias
09-zero-changes-external-cert-expiry
10-multi-region-asymmetry
11-capacity-bound-organic-growth
)
complete() { # exit 0 if fixture $1 has >=TRIALS in each arm
python - "$1" "$TRIALS" "$OUT" <<'PY'
import json, sys
fx, T, out = sys.argv[1], int(sys.argv[2]), sys.argv[3]
try:
d = json.load(open(out))
except Exception:
sys.exit(1)
c = sum(1 for r in d if r["incident"] == fx and r["condition"] == "control")
t = sum(1 for r in d if r["incident"] == fx and r["condition"] == "treatment")
print(f">>> {fx}: control {c}/{T}, treatment {t}/{T}")
sys.exit(0 if c >= T and t >= T else 1)
PY
}
echo "=== run_until_pass: TRIALS=$TRIALS, $((${#FIXTURES[@]})) fixtures ==="
for fx in "${FIXTURES[@]}"; do
echo "----- fixture $fx -----"
for attempt in $(seq 1 "$MAX_ATTEMPTS"); do
python -u tests/eval/run_eval.py --fixtures "$fx" --trials "$TRIALS" --output "$OUT" 2>&1 \
| grep -E 'investigate|ERROR on' || true
if complete "$fx"; then
echo " $fx COMPLETE (attempt $attempt)"
break
fi
echo " $fx incomplete after attempt $attempt; retrying..."
sleep 5
done
complete "$fx" >/dev/null || echo "!!! $fx still incomplete after $MAX_ATTEMPTS attempts; moving on"
done
echo "=== ALL FIXTURES PROCESSED — final summary ==="
python tests/eval/run_eval.py --trials "$TRIALS" --output "$OUT" 2>&1 | sed -n '/^Fixture/,$p'
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |