opensite-ai/opensite-skills / linux-ai-dev-stack
linux-ai-dev-stack
FlaggedInstall and wire the AI toolchain on a Linux dev machine: local LLM inference (Ollama, llama.cpp, vLLM, SGLang, TabbyAPI) with VRAM-based model selection, AI coding agents and CLIs (Claude Code, Codex, aider, OpenCode) with MCP servers, ML frameworks (PyTorch/JAX, Jupyter, HuggingFace) and GPU containers, and cloud/remote GPU workflows (RunPod, Vast.ai, Lambda, remote dev over SSH and Tailscale). Use when someone wants to run local models, set up or connect AI coding agents, configure MCP servers, route agents to local or cloud models, install an ML/GPU Python stack, or offload work to a rented GPU. It runs on the machine and assumes working GPU drivers.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.9 KB |
| agents/openai.yaml | config | 420 B |
| examples/24gb-nvidia-local-plus-cloud.md | markdown | 5.0 KB |
| references/activation.md | markdown | 2.9 KB |
| references/ai-coding-agents.md | markdown | 7.1 KB |
| references/local-llm-inference.md | markdown | 11.9 KB |
| references/mcp-servers.md | markdown | 4.5 KB |
| references/ml-frameworks.md | markdown | 5.0 KB |
| references/model-wiring.md | markdown | 4.6 KB |
| references/remote-gpu.md | markdown | 4.1 KB |
| templates/AGENTS.md | markdown | 1.8 KB |
| templates/gpu-compose.yaml | config | 1.4 KB |
| templates/litellm-config.yaml | config | 2.3 KB |
references/remote-gpu.md
View on GitHub# Remote / Cloud GPU
When local VRAM is not enough, rent a GPU: pick a provider, set up remote dev, sync
code and data, and connect over a private network. Provider pricing/features are
**as of 2026-07 — verify.**
## 1. When to go remote vs local
- **Local GPU** if you have a capable card (e.g. 24GB+ consumer GPU) and iterate
constantly — no per-hour cost, lowest latency, best privacy. Good for dev, small
models, LoRA, and inference.
- **Remote GPU** when you need big VRAM (A100/H100/B200), multi-GPU, short bursts, or
do not want to own hardware. Pay per second/minute: spin up, train, spin down.
## 2. Providers (2026 landscape)
| Provider | Best for | Notes |
|---|---|---|
| **RunPod** | **default choice** | Widest GPU catalog, **per-second** billing, Pods (VMs) + Serverless (scale-to-zero, ~<200ms cold starts on ~half of requests), one-click UI, network volumes. |
| **Vast.ai** | **cheapest raw compute** | Marketplace of independent hosts; RTX 4090 often ~$0.25–0.30/hr, ~40–60% cheaper, but variable reliability/bandwidth. Great for cost-driven, interruption-tolerant jobs. |
| **Lambda** | **multi-GPU training** | SXM clusters (8×H100/8×B200), **no egress fees**, predictable per-minute pricing, on-demand + reserved. |
| **Modal** | **serverless inference / Python-native** | Define GPU functions in Python, autoscaling, cron; often better than RunPod Serverless for inference. |
Others worth knowing: Paperspace, CoreWeave (enterprise), Fly.io GPUs, Together /
Replicate (hosted inference).
## 3. Remote dev tooling
- **SSH** is the foundation — hardened ed25519 keys, `~/.ssh/config` host aliases,
`ControlMaster` multiplexing for fast reconnects.
- **VS Code Remote-SSH / Dev Containers** — edits files on the remote with local UI;
extensions + language servers run remote. Best-in-class for remote GPU boxes.
- **Zed remote** (≥ v0.159) — direct SSH, headless server on the remote, local UI +
AI panel; fast alternative to VS Code Remote.
- **mosh** — resilient shell over UDP; survives roaming/flaky links and sleep/wake.
Pair with tmux/Zellij so long jobs persist across disconnects. (mosh has no
port-forwarding — keep SSH for that.)
- **Neovim over SSH + tmux** — the lowest-dependency remote workflow; works everywhere.
## 4. Syncing code & data
- **git** — canonical source of truth for code history (push/pull local ↔ remote).
- **rsync** — one-shot/scheduled sync of large artifacts, datasets, checkpoints
(`rsync -avzP --exclude .git`). Best for big files git should not hold.
- **mutagen** — continuous, low-latency two-way sync for edit-locally/run-remotely;
feels like a local filesystem without sshfs lag. Best for tight
iterate-on-remote-GPU loops.
- Recommendation: **git** for history, **mutagen** for live iteration, **rsync** for
datasets/checkpoints. Avoid sshfs for hot loops (latency).
## 5. Networking / tunneling
- **Tailscale** — WireGuard mesh VPN; connect laptop ↔ remote GPU ↔ home server by
stable MagicDNS names regardless of NAT/IP. SSH over Tailscale, ACLs, ephemeral
nodes for cloud instances. The easiest secure private networking for remote GPU
boxes — prefer it over opening public ports.
- **cloudflared** (Cloudflare Tunnel) — expose a service (Jupyter, an inference API)
via a public HTTPS hostname without opening ports; good for demos/webhooks. Add auth.
- Recommendation: **Tailscale** for private dev access; **cloudflared** only when you
must expose an endpoint publicly. The same LAN-exposure safety rules from
[local-llm-inference.md](local-llm-inference.md) apply to remote model servers.
## 6. Cost tips
- Prefer **per-second/per-minute** billing (RunPod) and **scale-to-zero serverless**
(RunPod Serverless / Modal) for spiky workloads.
- Use **spot/interruptible/Vast marketplace** for fault-tolerant training with
checkpointing.
- Watch **egress fees** (Lambda has none). Persist data on **network volumes** and
shut down compute when idle. Cache datasets/models on the volume to avoid
re-download.
- Right-size VRAM; use mixed precision/quantization (see
[local-llm-inference.md](local-llm-inference.md)).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 468k | Updated | Claude CodeCodex | Unverified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |