opensite-ai/opensite-skills / linux-ai-dev-stack
linux-ai-dev-stack
FlaggedInstall and wire the AI toolchain on a Linux dev machine: local LLM inference (Ollama, llama.cpp, vLLM, SGLang, TabbyAPI) with VRAM-based model selection, AI coding agents and CLIs (Claude Code, Codex, aider, OpenCode) with MCP servers, ML frameworks (PyTorch/JAX, Jupyter, HuggingFace) and GPU containers, and cloud/remote GPU workflows (RunPod, Vast.ai, Lambda, remote dev over SSH and Tailscale). Use when someone wants to run local models, set up or connect AI coding agents, configure MCP servers, route agents to local or cloud models, install an ML/GPU Python stack, or offload work to a rented GPU. It runs on the machine and assumes working GPU drivers.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.9 KB |
| agents/openai.yaml | config | 420 B |
| examples/24gb-nvidia-local-plus-cloud.md | markdown | 5.0 KB |
| references/activation.md | markdown | 2.9 KB |
| references/ai-coding-agents.md | markdown | 7.1 KB |
| references/local-llm-inference.md | markdown | 11.9 KB |
| references/mcp-servers.md | markdown | 4.5 KB |
| references/ml-frameworks.md | markdown | 5.0 KB |
| references/model-wiring.md | markdown | 4.6 KB |
| references/remote-gpu.md | markdown | 4.1 KB |
| templates/AGENTS.md | markdown | 1.8 KB |
| templates/gpu-compose.yaml | config | 1.4 KB |
| templates/litellm-config.yaml | config | 2.3 KB |
templates/litellm-config.yaml
View on GitHub# LiteLLM proxy config — routes local + cloud models behind one endpoint.
# Run: litellm --config litellm-config.yaml (default listen: http://localhost:4000)
# Docs (verify current schema, as of 2026-07): https://docs.litellm.ai/docs/proxy/configs
#
# Secrets: use os.environ/ references so keys never live in this file.
# Set ANTHROPIC_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY in your environment,
# ideally via a secrets manager (e.g. `op run -- litellm --config litellm-config.yaml`).
model_list:
# --- Anthropic-named alias mapped to a LOCAL model ---
# Claude Code speaks the Anthropic API; naming a local model after a real Anthropic
# model lets Claude Code use it unchanged. Point Claude Code here with:
# export ANTHROPIC_BASE_URL="http://localhost:4000"
# export ANTHROPIC_AUTH_TOKEN="sk-litellm-key"
- model_name: claude-3-5-sonnet-local
litellm_params:
model: ollama_chat/qwen2.5-coder:14b
api_base: http://localhost:11434
# Ollama's default context (2048/4096) truncates silently — raise it:
num_ctx: 32768
# --- Local models via their OpenAI-compatible endpoints ---
- model_name: local-qwen-coder
litellm_params:
model: openai/qwen2.5-coder:14b
api_base: http://localhost:11434/v1
api_key: "sk-no-key-required"
- model_name: local-vllm
litellm_params:
model: openai/Qwen/Qwen3-8B
api_base: http://localhost:8000/v1
api_key: "sk-no-key-required"
# --- Real cloud models (keys pulled from the environment) ---
- model_name: sonnet
litellm_params:
model: anthropic/claude-sonnet-4-5
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: gpt
litellm_params:
model: openai/gpt-5.1
api_key: os.environ/OPENAI_API_KEY
- model_name: gemini
litellm_params:
model: gemini/gemini-3-pro
api_key: os.environ/GEMINI_API_KEY
# Router behavior: fall back from local to a cloud model on error.
router_settings:
routing_strategy: simple-shuffle
num_retries: 2
fallbacks:
- claude-3-5-sonnet-local: ["sonnet"]
- local-qwen-coder: ["sonnet"]
# Proxy-level settings: require a virtual key and cap spend per key.
litellm_settings:
drop_params: true
set_verbose: false
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY # e.g. sk-... (set in the environment)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |