agentsope/SkillAlchemy / skills/agentsop-llm-engine-selection
agentsop-llm-engine-selection
VerifiedCross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 3.0 KB |
| SKILL.md | skill md | 24.2 KB |
| intermediate/operation_candidates.json | config | 7.5 KB |
| references/R1-source-evidence.md | markdown | 5.2 KB |
| references/R2-decision-flowchart.md | markdown | 5.7 KB |
README.md
View on GitHub# llm-engine-selection skill
Cross-engine decision rubric for **picking an LLM serving stack** — vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, or MLX — as a function of `(hardware × workload × constraint)`, not "which is fastest".
This is the **D5 gap skill** in the Phase-B coder-agent skill inventory. Individual engines have their own skills (`vllm`, `sglang`, `tensorrt-llm`, `llama-cpp`); this skill teaches the *cross-engine decision*.
## Scope
- **Activation**: any time a coder-agent picks, defends, or migrates an inference engine for production / batch / edge / dev / multi-tenant SaaS / structured-output workloads.
- **Decision axes**: hardware (GPU vendor + interconnect), workload (concurrency + ISL/OSL + prefix-share + structured-out), constraint (license + lock-in + eng-budget + SLA), model (family + size + quantization).
- **Date stamp**: May 2026. State-of-the-art moves in 3–6 month cycles; re-verify before committing.
## Layout
```
d-llm-engine-selection-skill/
├── SKILL.md # 7-section SOP (activation → cross-framework matrix)
├── README.md # This file
├── references/
│ ├── R1-source-evidence.md # Cited benchmarks and primary-source claims
│ └── R2-decision-flowchart.md # Decision tree + worked examples
└── intermediate/
└── operation_candidates.json # Raw trigger / action / output / evidence operations
```
## Key claim of the skill
> Engine choice is a function of `(hardware × workload × constraint)`, not "which is fastest".
Concretely, in 2026:
- **vLLM** owns the production default slot (vendor-neutral, broadest models, HF-blessed).
- **SGLang** wins on shared-prefix workloads (+~29% vs vLLM) and structured generation.
- **TensorRT-LLM** wins on saturated NVIDIA throughput (+30–50% vs vLLM), at the cost of 1–2 weeks setup + vendor lock-in.
- **TGI** is in maintenance mode; new projects in 2026 should not pick it.
- **llama.cpp / Ollama / MLX** own the CPU / Apple / edge / single-user slot.
The skill encodes how to map a constraint vector to one of these slots, plus 4 dilemma cases (PCIe-only vLLM box, structured-JSON-at-scale, NVIDIA shop with 2-week budget, laptop-to-cloud parity) and 7 anti-patterns.
## Method
Primary mining: the `vllm-sop-skill/references/R5-ecosystem-context.md` source-of-truth comparison plus frontmatter from the four engine skills (`vllm`, `sglang`, `tensorrt-llm`, `llama-cpp`) under `~/.claude/skills/`. Every load-bearing performance claim carries an inline `[source]` tag and resolves to the cited-sources list. No fabrication.
## Position in the Phase-B inventory
- **Companion skills (per-engine deep dives)**: `vllm`, `sglang`, `tensorrt-llm`, `llama-cpp`.
- **Sibling decision skill**: `vllm-sop` covers the same selection logic from the vLLM-centric view; this skill is engine-agnostic and is the right activation when the choice is genuinely open.
- **Date sensitivity**: Phase-B medium-frequency, specialized. Re-skill every 2 quarters.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |