agentsope/SkillAlchemy / skills/agentsop-vllm
agentsop-vllm
VerifiedDecision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 2.4 KB |
| SKILL.md | skill md | 24.1 KB |
| intermediate/operation_candidates.json | config | 6.9 KB |
| references/R1-architecture.md | markdown | 5.1 KB |
| references/R2-sop-workflow.md | markdown | 6.4 KB |
| references/R3-dilemma-cases.md | markdown | 7.6 KB |
| references/R4-anti-patterns.md | markdown | 6.3 KB |
| references/R5-ecosystem-context.md | markdown | 4.7 KB |
README.md
View on GitHub# vllm-sop skill
Decision SOP for serving LLMs with **vLLM** — extracted as a tool skill for coder-agent use.
This is one of seven projects in a horizontal landscape study of LLM-app frameworks (LangGraph, LlamaIndex, DSPy, CrewAI, vLLM, Aider, Dify). The goal across the set is to capture each tool's **operating system** — its decision patterns, when-to-use logic, and dilemma resolution — rather than its API surface.
## Scope
- **Domain**: high-throughput LLM inference serving on GPU.
- **Anchors**: PagedAttention (Kwon et al., SOSP 2023, arxiv 2309.06180), continuous batching (Orca-style iteration-level scheduling), prefix caching, tensor/pipeline/expert parallelism, quantization (FP8/AWQ/GPTQ), speculative decoding.
- **Activation**: when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.
## Layout
```
vllm-sop-skill/
├── SKILL.md # Main SOP — 7 sections (activation → ecosystem)
├── README.md # This file
├── references/
│ ├── R1-architecture.md # PagedAttention, scheduler, prefill/decode split
│ ├── R2-sop-workflow.md # 7-step deploy workflow with decision points
│ ├── R3-dilemma-cases.md # 5 cited dilemma cases
│ ├── R4-anti-patterns.md # 7 anti-patterns + boundary conditions
│ └── R5-ecosystem-context.md # vLLM vs TGI/SGLang/TensorRT-LLM/llama.cpp/Ollama
└── intermediate/
└── operation_candidates.json # Raw trigger/action/output/evidence operations
```
## Method
Source mining used WebSearch + WebFetch against: the PagedAttention paper, docs.vllm.ai, the vLLM GitHub README, Red Hat tuning guides, comparison benchmarks (2025–2026 vintage), and a quantization-tradeoffs preprint (arxiv 2411.02355). Every load-bearing claim in SKILL.md carries an inline `[source]` tag. No fabrication.
## Surprises worth noting
1. The bottleneck was never compute — it was KV-cache fragmentation. Pre-vLLM engines used 20–40% of GPU memory.
2. TP=4 across PCIe is often slower than 2 independent TP=2 replicas. Interconnect topology beats parallelism degree.
3. Speculative decoding can make things 175% worse with bad hyperparameters, despite the up-to-2.5× best-case headline.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |