opensite-ai/opensite-skills / aws-compute
aws-compute
VerifiedProvisions, scales, and operates Amazon EC2 virtual-machine workloads: instance-type selection (Graviton/Arm64, burstable T credits, GPU, instance store vs EBS), launch templates, Auto Scaling groups (scaling policies, instance refresh, mixed instances, Spot, warm pools, lifecycle hooks), IMDSv2, placement groups, Elastic IPs, AMI lifecycle, and Systems Manager fleet operations (Session Manager, Run Command, Patch Manager). Applies to EC2 instance and fleet questions, InsufficientInstanceCapacity, CPU-credit/surplus charges, IMDSv2 401s, instances stuck in Pending:Wait, ASG not replacing unhealthy instances, status-check failures, SSH refused/timed out, or instances missing as SSM managed nodes. For a single secure instance launch, the launching-ec2-instance-with-best-practices skill is more appropriate; for instance profiles, see setting-up-ec2-instance-profiles; for Image Builder, see amazon-ec2-image-builder. Does NOT cover Lambda, ECS/Fargate, EKS, VPC/ALB/NLB design, or IAM policy authoring.
| File | Kind | Size |
|---|---|---|
| .aws-skill-metadata | other | 18 B |
| SKILL.md | skill md | 8.7 KB |
| references/ami-management.md | markdown | 5.9 KB |
| references/auto-scaling.md | markdown | 10.1 KB |
| references/instance-selection.md | markdown | 5.7 KB |
| references/provisioning.md | markdown | 9.2 KB |
| references/systems-manager.md | markdown | 7.4 KB |
| references/troubleshooting.md | markdown | 5.7 KB |
references/troubleshooting.md
View on GitHub# EC2 Troubleshooting
## Overview
Diagnosing the common EC2 failures: connectivity, status checks, capacity/quota errors, Spot interruptions, credit exhaustion, and instances that die on launch. Match the exact error string first — several look similar but have opposite fixes.
## Can't SSH in: read the exact error
The error string tells you the layer.
| Error | Layer | Causes | Fix |
|-------|-------|--------|-----|
| **Connection timed out** | Network — packets never arrive | Security group missing inbound TCP 22 from your IP; NACL blocking; no route to internet gateway; instance has no/changed public IP; local firewall | Open 22 from your IP; verify route + public IP; note public IP changes on stop/start unless an Elastic IP is attached |
| **Connection refused** | Host — reachable, nothing on the port | sshd not running/listening on that port; instance still booting; sshd misconfigured | Wait for boot; check sshd via Session Manager or EC2 serial console |
| **Permission denied (publickey)** | Auth | Wrong username for the AMI; wrong key; key file perms; EC2 Instance Connect key expired (60s) | Use the AMI's default user (ec2-user/ubuntu/admin/root); `chmod 400 key.pem`; re-run send-ssh-public-key |
Keyless fallbacks: Session Manager, EC2 Instance Connect, or the serial console (Nitro instances). The serial console is **not on by default** — it needs account-level enablement plus IAM permissions, and on Linux requires a preconfigured password-based user (only the browser client itself is truly keyless). The `AWSSupport-TroubleshootSSH` runbook automates checks. Debug with `ssh -vvv -i key.pem user@host`. For "Connection timed out", enable VPC Flow Logs to confirm whether packets reach the ENI (ACCEPT/REJECT) — KMS-encrypt the flow-log destination (S3 bucket via SSE-KMS or CloudWatch Logs log group); add the CloudWatch agent for OS-level visibility.
## Status check failed: system vs instance
| Check | Monitors | Remedy |
|-------|----------|--------|
| **System** (`StatusCheckFailed_System`) | AWS host/hardware/power/network | **Stop/start** an EBS-backed instance to migrate to new hardware (a reboot stays on the same host); or set a CloudWatch instance-recovery action; or wait for AWS |
| **Instance** (`StatusCheckFailed_Instance`) | The instance's OS/network config | Reboot or fix the OS/network config yourself |
| Attached EBS | Attached EBS volumes | Investigate the volume; ASG replacement isn't automatic for this check — by default. Enabling the opt-in EBS health check on the ASG (Nitro instances) makes it detect attached-EBS impairment and replace the instance automatically |
## Capacity vs quota errors (opposite fixes)
| Error | Meaning | Fix |
|-------|---------|-----|
| **InsufficientInstanceCapacity** | AWS lacks spare capacity for that type in that AZ — NOT a quota | Try another AZ, another instance type/size, retry later, or request fewer per call. In a cluster placement group, stop/start all group instances to re-place them |
| **InstanceLimitExceeded** | You hit an On-Demand vCPU-based quota (grouped, e.g. the Standard family group A/C/D/H/I/M/R/T/Z; the vCPU-phrased variant is `VcpuLimitExceeded`) — this IS a quota | Raise it via Service Quotas for that quota and Region |
A quota increase does nothing for `InsufficientInstanceCapacity`.
## Spot interruptions
Spot instances get a **2-minute interruption notice** (via instance metadata `instance-action` and the EventBridge event `EC2 Spot Instance Interruption Warning`). An earlier, softer signal — the **rebalance recommendation** (`EC2 Instance Rebalance Recommendation`) — usually arrives earlier than (but is best-effort and can arrive together with) the two-minute interruption notice, giving more time to drain. In an ASG, enable Capacity Rebalancing to act on it automatically (see [auto-scaling.md](auto-scaling.md)). Test interruption handling with AWS Fault Injection Service.
## High CPU / sudden slowdown on burstable (T) instances
A T instance that suddenly and persistently drops to sluggish performance under sustained load has likely exhausted its CPU credits: in `standard` mode it throttles to baseline. Switch to `unlimited` (watch surplus charges) or move to a fixed-performance family. In `unlimited` mode the reverse symptom — a surprise bill — comes from `CPUSurplusCreditsCharged`. See [instance-selection.md](instance-selection.md).
## Instance immediately stops or terminates after launch
- **Encrypted root EBS volume the launch role can't decrypt** — the principal/role lacks KMS key permissions; grant key access. Check the system log to confirm the OS never booted.
- Hitting EBS volume quotas can block/impair launches.
- Remember instance store data is gone after any stop.
## Instance status check fails after changing to a Nitro type
Migrating to a Nitro-based instance type without the ENA and NVMe drivers installed causes instance status check failures — install the drivers before switching.
## Security Considerations
- **Prefer Session Manager over SSH for diagnosis** — keyless, IAM-scoped, and fully audited via CloudTrail (no key sprawl or open port 22).
- **Treat repeated connection failures as possible reconnaissance** — enable CloudTrail and VPC Flow Logs so failed access attempts are visible.
- **A status-check failure can be a security event, not just hardware** — before reflexively rebooting a suspect instance, review system logs and consider isolation/forensics (a reboot can destroy volatile evidence and instance-store data).
## Related
- [auto-scaling.md](auto-scaling.md) — ASG health, Spot, lifecycle hooks
- [provisioning.md](provisioning.md) — Elastic IPs, key pairs, IMDS
- [systems-manager.md](systems-manager.md) — keyless access as an SSH fallback
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |