Skip to content

agentsope/SkillAlchemy / skills/agentsop-vllm

agentsop-vllm

Verified

Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

# vllm-sop skill

Decision SOP for serving LLMs with **vLLM** — extracted as a tool skill for coder-agent use.

This is one of seven projects in a horizontal landscape study of LLM-app frameworks (LangGraph, LlamaIndex, DSPy, CrewAI, vLLM, Aider, Dify). The goal across the set is to capture each tool's **operating system** — its decision patterns, when-to-use logic, and dilemma resolution — rather than its API surface.

## Scope

- **Domain**: high-throughput LLM inference serving on GPU.
- **Anchors**: PagedAttention (Kwon et al., SOSP 2023, arxiv 2309.06180), continuous batching (Orca-style iteration-level scheduling), prefix caching, tensor/pipeline/expert parallelism, quantization (FP8/AWQ/GPTQ), speculative decoding.
- **Activation**: when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

## Layout

```
vllm-sop-skill/
├── SKILL.md                          # Main SOP — 7 sections (activation → ecosystem)
├── README.md                         # This file
├── references/
│   ├── R1-architecture.md            # PagedAttention, scheduler, prefill/decode split
│   ├── R2-sop-workflow.md            # 7-step deploy workflow with decision points
│   ├── R3-dilemma-cases.md           # 5 cited dilemma cases
│   ├── R4-anti-patterns.md           # 7 anti-patterns + boundary conditions
│   └── R5-ecosystem-context.md       # vLLM vs TGI/SGLang/TensorRT-LLM/llama.cpp/Ollama
└── intermediate/
    └── operation_candidates.json     # Raw trigger/action/output/evidence operations
```

## Method

Source mining used WebSearch + WebFetch against: the PagedAttention paper, docs.vllm.ai, the vLLM GitHub README, Red Hat tuning guides, comparison benchmarks (2025–2026 vintage), and a quantization-tradeoffs preprint (arxiv 2411.02355). Every load-bearing claim in SKILL.md carries an inline `[source]` tag. No fabrication.

## Surprises worth noting

1. The bottleneck was never compute — it was KV-cache fragmentation. Pre-vLLM engines used 20–40% of GPU memory.
2. TP=4 across PCIe is often slower than 2 independent TP=2 replicas. Interconnect topology beats parallelism degree.
3. Speculative decoding can make things 175% worse with bad hyperparameters, despite the up-to-2.5× best-case headline.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall