Skip to content

davila7/claude-code-templates / cli-tool/components/skills/ai-research/loki-mode

loki-mode

Flagged

Multi-agent autonomous startup system for Claude Code. Triggers on "Loki Mode". Orchestrates 100+ specialized agents across engineering, QA, DevOps, security, data/ML, business operations, marketing, HR, and customer success. Takes PRD to fully deployed, revenue-generating product with zero human intervention. Features Task tool for subagent dispatch, parallel code review with 3 specialized reviewers, severity-based issue triage, distributed task queue with dead letter handling, automatic deployment to cloud providers, A/B testing, customer feedback loops, incident response, circuit breakers, and self-healing. Handles rate limits via distributed state checkpoints and auto-resume with exponential backoff. Requires --dangerously-skip-permissions flag.

FileKindSize
.github/workflows/claude-code-review.ymlconfig1.9 KB
.github/workflows/claude.ymlconfig1.8 KB
.github/workflows/release.ymlconfig4.8 KB
.gitignoreother10 B
ACKNOWLEDGEMENTS.mdmarkdown11.8 KB
CHANGELOG.mdmarkdown83.9 KB
CLAUDE.mdmarkdown4.3 KB
CONTEXT-EXPORT.mdmarkdown6.5 KB
INSTALLATION.mdmarkdown9.1 KB
LICENSEother1.1 KB
README.mdmarkdown18.2 KB
SKILL.mdskill md27.9 KB
VERSIONother7 B
autonomy/.loki/dashboard/index.htmlother15.2 KB
autonomy/CONSTITUTION.mdmarkdown10.2 KB
autonomy/README.mdmarkdown7.5 KB
autonomy/run.shexecutablescript74.4 KB
benchmarks/datasets/humaneval.jsonlother209.4 KB
benchmarks/datasets/swebench-lite.jsonconfig321 B
benchmarks/prepare-submission.shexecutablescript6.6 KB
benchmarks/results/2026-01-05-00-23-56/SUMMARY.mdmarkdown1.2 KB
benchmarks/results/2026-01-05-00-23-56/humaneval-results.jsonconfig365 B
benchmarks/results/2026-01-05-00-23-56/swebench-results.jsonconfig383 B
benchmarks/results/2026-01-05-00-49-17/SUMMARY.mdmarkdown1.2 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-results.jsonconfig33.6 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/0.pyscript528 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/1.pyscript936 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/10.pyscript758 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/100.pyscript598 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/101.pyscript537 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/102.pyscript436 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/103.pyscript519 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/104.pyscript602 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/105.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/106.pyscript749 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/107.pyscript938 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/108.pyscript720 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/109.pyscript1.6 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/11.pyscript443 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/110.pyscript882 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/111.pyscript859 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/112.pyscript732 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/113.pyscript853 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/114.pyscript447 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/115.pyscript1.2 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/116.pyscript521 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/117.pyscript995 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/118.pyscript835 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/119.pyscript1002 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/12.pyscript564 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/120.pyscript706 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/121.pyscript339 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/122.pyscript494 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/123.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/124.pyscript1.6 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/125.pyscript757 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/126.pyscript943 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/127.pyscript1.5 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/128.pyscript653 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/129.pyscript2.2 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/13.pyscript264 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/130.pyscript940 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/131.pyscript417 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/132.pyscript1.0 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/133.pyscript524 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/134.pyscript779 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/135.pyscript481 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/136.pyscript667 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/137.pyscript751 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/138.pyscript295 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/139.pyscript455 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/14.pyscript263 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/140.pyscript989 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/141.pyscript1.4 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/142.pyscript822 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/143.pyscript1.0 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/144.pyscript780 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/145.pyscript702 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/146.pyscript700 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/147.pyscript734 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/148.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/149.pyscript880 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/15.pyscript266 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/150.pyscript371 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/151.pyscript647 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/152.pyscript845 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/153.pyscript1.4 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/154.pyscript567 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/155.pyscript427 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/156.pyscript750 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/157.pyscript463 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/158.pyscript524 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/159.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/16.pyscript295 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/160.pyscript968 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/161.pyscript637 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/162.pyscript337 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/163.pyscript425 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/17.pyscript736 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/18.pyscript548 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/19.pyscript714 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/2.pyscript360 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/20.pyscript835 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/21.pyscript493 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/22.pyscript349 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/23.pyscript153 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/24.pyscript260 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/25.pyscript689 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/26.pyscript406 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/27.pyscript205 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/28.pyscript240 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/29.pyscript367 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/3.pyscript590 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/30.pyscript262 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/31.pyscript525 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/32.pyscript1.5 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/33.pyscript710 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/34.pyscript172 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/35.pyscript200 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/36.pyscript357 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/37.pyscript481 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/38.pyscript567 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/39.pyscript762 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/4.pyscript530 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/40.pyscript684 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/41.pyscript661 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/42.pyscript239 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/43.pyscript582 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/44.pyscript441 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/45.pyscript158 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/46.pyscript620 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/47.pyscript342 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/48.pyscript271 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/49.pyscript245 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/5.pyscript466 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/50.pyscript412 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/51.pyscript469 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/52.pyscript255 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/53.pyscript129 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/54.pyscript458 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/55.pyscript302 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/56.pyscript559 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/57.pyscript458 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/58.pyscript268 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/59.pyscript477 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/6.pyscript832 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/60.pyscript259 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/61.pyscript559 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/62.pyscript333 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/63.pyscript624 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/64.pyscript560 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/65.pyscript433 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/66.pyscript410 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/67.pyscript876 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/68.pyscript1.5 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/69.pyscript709 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/7.pyscript379 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/70.pyscript693 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/71.pyscript589 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/72.pyscript837 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/73.pyscript612 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/74.pyscript779 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/75.pyscript846 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/76.pyscript571 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/77.pyscript467 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/78.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/79.pyscript600 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/8.pyscript527 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/80.pyscript563 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/81.pyscript1.9 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/82.pyscript580 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/83.pyscript855 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/84.pyscript556 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/85.pyscript288 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/86.pyscript660 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/87.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/88.pyscript784 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/89.pyscript847 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/9.pyscript506 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/90.pyscript531 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/91.pyscript601 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/92.pyscript586 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/93.pyscript974 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/94.pyscript1.1 KB
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/95.pyscript1003 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/96.pyscript804 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/97.pyscript421 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/98.pyscript396 B
benchmarks/results/2026-01-05-00-49-17/humaneval-solutions/99.pyscript886 B
benchmarks/results/2026-01-05-01-24-17/SUMMARY.mdmarkdown1.3 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/astropy__astropy-12907.patchother1.0 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/astropy__astropy-14182.patchother2.3 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/astropy__astropy-14365.patchother1.3 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/astropy__astropy-14995.patchother1.1 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/astropy__astropy-6938.patchother997 B
benchmarks/results/2026-01-05-01-24-17/swebench-patches/astropy__astropy-7746.patchother1.6 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/django__django-10914.patchother3.3 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/django__django-10924.patchother1.9 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/django__django-11001.patchother4.3 KB
benchmarks/results/2026-01-05-01-24-17/swebench-patches/django__django-11019.patchother20.2 KB

benchmarks/prepare-submission.sh

View on GitHub
#!/bin/bash
#===============================================================================
# Prepare SWE-bench Submission
# Converts benchmark results to official SWE-bench submission format
#
# Usage:
#   ./benchmarks/prepare-submission.sh <results-dir>
#   ./benchmarks/prepare-submission.sh benchmarks/results/2026-01-05-10-37-54
#
# Output:
#   Creates submission-ready folder at benchmarks/submission/
#===============================================================================

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"

# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
CYAN='\033[0;36m'
NC='\033[0m'

log_info() { echo -e "${CYAN}[INFO]${NC} $1"; }
log_success() { echo -e "${GREEN}[PASS]${NC} $1"; }
log_error() { echo -e "${RED}[FAIL]${NC} $1"; }

if [ $# -lt 1 ]; then
    echo "Usage: $0 <results-directory>"
    echo "Example: $0 benchmarks/results/2026-01-05-10-37-54"
    exit 1
fi

RESULTS_DIR="$1"
SUBMISSION_DATE=$(date +%Y%m%d)
SUBMISSION_DIR="$SCRIPT_DIR/submission/${SUBMISSION_DATE}_loki_mode"

log_info "Preparing SWE-bench submission..."
log_info "Results: $RESULTS_DIR"
log_info "Output: $SUBMISSION_DIR"

# Check results directory
if [ ! -d "$RESULTS_DIR" ]; then
    log_error "Results directory not found: $RESULTS_DIR"
    exit 1
fi

# Check for required files
if [ ! -f "$RESULTS_DIR/swebench-loki-predictions.json" ]; then
    log_error "Predictions file not found: $RESULTS_DIR/swebench-loki-predictions.json"
    exit 1
fi

# Create submission directory
mkdir -p "$SUBMISSION_DIR"

# Copy template files
log_info "Copying template files..."
cp "$SCRIPT_DIR/submission-template/README.md" "$SUBMISSION_DIR/"
cp "$SCRIPT_DIR/submission-template/metadata.yaml" "$SUBMISSION_DIR/"

# Convert predictions to JSONL format
log_info "Converting predictions to JSONL format..."
python3 << CONVERT_PREDS
import json

with open("$RESULTS_DIR/swebench-loki-predictions.json", 'r') as f:
    predictions = json.load(f)

with open("$SUBMISSION_DIR/all_preds.jsonl", 'w') as f:
    for pred in predictions:
        # Format required by SWE-bench
        entry = {
            "instance_id": pred["instance_id"],
            "model_patch": pred["model_patch"],
            "model_name_or_path": pred.get("model_name_or_path", "loki-mode")
        }
        f.write(json.dumps(entry) + '\n')

print(f"Converted {len(predictions)} predictions to JSONL format")
CONVERT_PREDS

# Copy trajectories if they exist
if [ -d "$RESULTS_DIR/trajs" ]; then
    log_info "Copying trajectory files..."
    cp -r "$RESULTS_DIR/trajs" "$SUBMISSION_DIR/"
    TRAJ_COUNT=$(ls -1 "$SUBMISSION_DIR/trajs" 2>/dev/null | wc -l | tr -d ' ')
    log_success "Copied $TRAJ_COUNT trajectory files"
else
    log_info "No trajectory files found (run benchmark with --loki for trajectory logging)"
    mkdir -p "$SUBMISSION_DIR/trajs"
fi

# Copy logs if they exist
if [ -d "$RESULTS_DIR/logs" ]; then
    log_info "Copying log files..."
    cp -r "$RESULTS_DIR/logs" "$SUBMISSION_DIR/"
    LOG_COUNT=$(ls -1 "$SUBMISSION_DIR/logs" 2>/dev/null | wc -l | tr -d ' ')
    log_success "Copied $LOG_COUNT log directories"
else
    log_info "No log files found (run benchmark with --loki for log capture)"
    mkdir -p "$SUBMISSION_DIR/logs"
fi

# Update metadata with actual results
log_info "Updating metadata with actual results..."
python3 << UPDATE_META
import json
import yaml
from datetime import datetime

# Load results
with open("$RESULTS_DIR/swebench-loki-results.json", 'r') as f:
    results = json.load(f)

# Load metadata template
with open("$SUBMISSION_DIR/metadata.yaml", 'r') as f:
    metadata = yaml.safe_load(f)

# Update with actual results
metadata['results'] = {
    'patch_generation_rate': round((results.get('generated', 0) / results.get('total_problems', 1)) * 100, 2),
    'problems_solved': results.get('generated', 0),
    'problems_total': results.get('total_problems', 0),
    'fixed_by_rarv': results.get('fixed_by_rarv', 0),
    'avg_attempts': round(results.get('avg_attempts', 1.0), 2),
    'total_time_seconds': round(results.get('elapsed_time', 0)),
    'avg_time_per_problem_seconds': round(results.get('elapsed_time', 0) / max(results.get('total_problems', 1), 1))
}
metadata['submission']['date'] = datetime.now().strftime('%Y-%m-%d')

# Save updated metadata
with open("$SUBMISSION_DIR/metadata.yaml", 'w') as f:
    yaml.dump(metadata, f, default_flow_style=False, sort_keys=False)

print("Metadata updated with actual results")
CONVERT_PREDS

# Generate submission summary
log_info "Generating submission summary..."
cat > "$SUBMISSION_DIR/SUBMISSION_CHECKLIST.md" << 'CHECKLIST'
# SWE-bench Submission Checklist

## Required Files
- [x] all_preds.jsonl - Predictions in JSONL format
- [x] README.md - Description of the system
- [x] metadata.yaml - Submission metadata

## Optional but Recommended
- [ ] trajs/ - Reasoning trajectories (required for some leaderboards)
- [ ] logs/ - Execution logs

## Pre-Submission Steps

1. **Verify predictions format:**
   ```bash
   head -1 all_preds.jsonl | python -m json.tool
   ```

2. **Run SWE-bench evaluator (optional but recommended):**
   ```bash
   python -m swebench.harness.run_evaluation \
     --predictions all_preds.jsonl \
     --max_workers 4 \
     --run_id loki_mode_v2.25.0
   ```

3. **Fork and create PR:**
   ```bash
   # Fork https://github.com/SWE-bench/experiments
   # Clone your fork
   git clone https://github.com/YOUR_USERNAME/experiments.git
   cd experiments

   # Copy submission
   cp -r /path/to/submission evaluation/lite/20260105_loki_mode

   # Create PR
   git checkout -b loki-mode-submission
   git add .
   git commit -m "Add Loki Mode submission"
   git push origin loki-mode-submission
   ```

4. **Submit PR with:**
   - Link to this repository
   - Brief description of the system
   - Any relevant benchmark methodology notes

## Contact

For questions about this submission, open an issue at:
https://github.com/asklokesh/loki-mode/issues
CHECKLIST

# Final summary
echo ""
echo "======================================================================"
echo "  SUBMISSION PREPARED"
echo "======================================================================"
echo "  Location: $SUBMISSION_DIR"
echo ""
echo "  Files:"
ls -la "$SUBMISSION_DIR/"
echo ""
echo "  Next Steps:"
echo "  1. Review all_preds.jsonl format"
echo "  2. Run SWE-bench evaluator (optional)"
echo "  3. Fork SWE-bench/experiments"
echo "  4. Copy submission folder to evaluation/lite/"
echo "  5. Create pull request"
echo "======================================================================"

log_success "Submission preparation complete!"
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall