K-Dense-AI/scientific-agent-skills / skills/get-available-resources
get-available-resources
VerifiedDetects host inventory and effective CPU, memory, disk, scheduler, container, and accelerator limits when a user asks for resource-aware planning or before a clearly resource-sensitive local workload. Produces a redacted JSON snapshot and conservative planning helpers without stress tests or assuming visible host hardware is usable.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 11.4 KB |
| references/resource_semantics.md | markdown | 10.9 KB |
| references/snapshot_schema.md | markdown | 6.4 KB |
| references/sources.md | markdown | 8.2 KB |
| scripts/_common.py | script | 6.4 KB |
| scripts/accelerator_diagnostics.py | script | 4.5 KB |
| scripts/detect_resources.py | script | 64.6 KB |
| scripts/plan_workload.py | script | 10.9 KB |
| scripts/snapshot_tools.py | script | 17.6 KB |
references/resource_semantics.md
View on GitHub# Resource Semantics
Research and behavior cut-off: **2026-10-01**. See
[`sources.md`](sources.md) for the official documentation used.
## Core rule: inventory is not entitlement
Never treat a host-wide count as a promise that the current process can use it.
Interpret usable resources as the intersection of independently observed
constraints:
1. host inventory;
2. process affinity or processor-group scope;
3. cgroup/container constraints;
4. scheduler allocation;
5. accelerator visibility and device permissions; and
6. application runtime compatibility.
Missing evidence means **unknown**, not unlimited. Scheduler variables can
describe an allocation without proving that affinity or cgroup enforcement is
enabled. Conversely, a cgroup or affinity mask can be stricter than the
scheduler request.
## CPU
### Physical and logical CPUs
- A logical CPU is an operating-system scheduling unit. Simultaneous
multithreading can expose multiple logical CPUs on one physical core.
- A physical-core count describes topology, not the number of independent
workers the process may start.
- `psutil.cpu_count(logical=True)`, macOS `hw.logicalcpu`, and POSIX
`SC_NPROCESSORS_ONLN` provide host/system inventory. They can exceed the
CPUs usable by the process. `os.cpu_count()` is a fallback: Python 3.13+
`-X cpu_count` or `PYTHON_CPU_COUNT` overrides both Python CPU-count APIs.
The detector prefers independent inventory and keeps Python process count
as a separate planning input.
- `os.process_cpu_count()` (Python 3.13+) is process-aware. On supported
platforms, affinity APIs provide a more explicit process constraint.
- On Windows systems with multiple processor groups, a system-wide logical
count and one process/thread group's usable count can differ. Windows 11
and Server 2022 default affinity spans groups; do not assume every modern
Windows process is capped at 64 logical CPUs.
The detector reports host logical and physical counts separately. It never
derives physical cores from a logical count.
### Affinity, cpusets, and quotas
- Process affinity limits the logical CPUs on which a process may execute.
- Linux `cpuset.cpus.effective` reports CPUs actually granted after parent
constraints. The requested `cpuset.cpus` can differ.
- `/proc/self/status` exposes `Cpus_allowed_list`, but the detector prefers
affinity APIs and cgroup effective cpusets.
- cgroup v2 `cpu.max` is `$MAX $PERIOD`. `max` means no local bandwidth limit.
A finite ratio is CPU-time capacity, possibly fractional; it is not a core
topology count.
- Parent cgroups also constrain children, so the detector takes the most
restrictive finite visible ancestor quota. Membership is resolved through
`/proc/self/mountinfo` cgroup2 root/mountpoint fields. A namespace or delegated
mount may hide stronger ancestors. Missing files are not evidence of
unlimited capacity; cgroup v1 is outside the collector's implemented scope.
For a `cpu.max` ratio of 1.5, the snapshot reports
`capacity_cores: 1.5` and a conservative CPU-bound worker ceiling of 1. A
workload may choose more threads for latency hiding, but it should expect
throttling and must not describe those threads as 1.5 physical cores.
### Python worker pools
- Current Python `multiprocessing.Pool` and `ProcessPoolExecutor` defaults use
`os.process_cpu_count()` when available.
- Python 3.14 no longer uses `fork` as the default start method on any platform.
Code that depends on a particular start method must request it deliberately.
- Process workers do not make each worker's native-library threads disappear.
BLAS/OpenMP threads multiplied by process workers commonly oversubscribe an
allocation.
- Windows `ProcessPoolExecutor` has a documented maximum of 61 workers.
Use the workload planner's worker and threads-per-worker values as conservative
ceilings, then benchmark a real representative workload. Do not run a synthetic
stress test merely to discover capacity.
## Memory
### Host and effective memory
- `psutil.virtual_memory().total` and `.available` describe system-visible
memory, not necessarily the process limit.
- Linux `/proc/meminfo` `MemAvailable` is the standard-library fallback for a
host-availability estimate.
- cgroup v2 `memory.current` is current cgroup usage.
- `memory.high` is a throttle/reclaim pressure boundary. Exceeding it does not
itself invoke the cgroup OOM killer, and the value can be breached.
- `memory.max` is the hard cgroup limit. If usage cannot be reduced at that
boundary, the cgroup OOM killer can run.
- Parent memory limits are hierarchical. Shared ancestor usage can reduce what
remains for a child, so effective remaining memory is the minimum finite
`limit - current` observed through the ancestor chain.
- Slurm memory variables describe requested/allocated memory, but strict
enforcement depends on site configuration.
The detector keeps host total/available values, cgroup values, scheduler values,
and the conservative effective values distinct. A point-in-time "available"
value can change immediately and is not a reservation.
### macOS unified memory
On Apple silicon, CPU and integrated GPU share unified memory. Do not add a
fictional GPU VRAM amount to system RAM. The snapshot marks the memory model as
`unified_cpu_gpu` and leaves dedicated GPU memory null. Metal framework support
and model/operator support still require application-specific checks.
## Accelerators
Accelerator usability has separate layers:
1. **Hardware or management visibility** — a management query returns a device.
2. **Allocation visibility** — scheduler and named visibility variables permit
a device or a subset.
3. **Device permission** — the process/container can open the required device
interfaces.
4. **Driver/runtime compatibility** — driver, CUDA/ROCm/Metal runtime, and
framework versions are compatible.
5. **Workload compatibility** — the requested operation and data type are
implemented on that backend.
`nvidia-smi` success confirms NVIDIA management visibility only. It does not
prove that CUDA libraries exist or are compatible. NVIDIA documents driver and
runtime compatibility as a separate requirement.
AMD SMI or ROCm SMI success likewise does not prove HIP/ROCm runtime usability.
New deployments should prefer `amd-smi`; `rocm-smi` is retained as a read-only
fallback. The fixed AMD query selects `--asic --vram`, avoiding unrelated
static fields. Current JSON nests VRAM size as `{value, unit}`; legacy ROCm SMI
uses a flat byte field. AMD SMI labels binary MiB values `MB`, confirmed in its
source conversion. The detector exposes this as `reported_total_bytes` and
leaves dedicated memory null because APU/shared-memory topology is unresolved.
On Linux, AMD recommends `ROCR_VISIBLE_DEVICES`; on Windows it recommends
`HIP_VISIBLE_DEVICES`.
An Apple integrated GPU is a Metal candidate, not a CUDA GPU. AMD SMI records
are ROCm candidates; macOS display records are Metal candidates regardless of
hardware vendor. Neural engines, TPUs, FPGAs, and other accelerators
must also remain distinct from CUDA devices if another inventory source adds
them.
The detector reads only these accelerator variable names and redacts their
values:
- `NVIDIA_VISIBLE_DEVICES`
- `CUDA_VISIBLE_DEVICES`
- `ROCR_VISIBLE_DEVICES`
- `HIP_VISIBLE_DEVICES`
Counts derived from those values are only upper bounds. Environment variables
are not a security boundary and can be reset by an application; device
namespace/cgroup controls are stronger isolation.
Management counts describe records in a particular query. A physical NVIDIA
GPU can expose several MIG devices, and AMD partition modes can also change
runtime enumeration. Use the workload framework in its actual allocation to
count usable devices; do not infer it from these candidate counts.
## Slurm and other schedulers
The detector allowlists named Slurm variables and never dumps the environment.
It records field names, parsed bounded counts, and memory quantities, but not
job IDs, GPU UUIDs, node names, submit hosts, or paths.
Important scopes:
- `SLURM_CPUS_PER_TASK`: requested CPUs per task; suitable as a per-process
upper bound for one task process.
- `SLURM_CPUS_ON_NODE`: CPUs allocated to the current batch step on the node;
it can be shared among tasks.
- `SLURM_JOB_CPUS_PER_NODE`: a per-node allocation list, not a process count.
- `SLURM_TASKS_PER_NODE`: compressed counts such as `1(x2),4`; resolve the
current entry with `SLURM_NODEID`. Without that index, only a uniform list
is interpretable. Never substitute the first node's task count. The node ID
itself is not emitted. Requested `SLURM_NTASKS_PER_NODE` is a fallback only
when the allocation list is absent. Step overrides and heterogeneous job
components still require application/site-specific interpretation.
- `SLURM_MEM_PER_CPU`: memory per allocated CPU. It becomes a per-task bound
only when CPUs per task is known.
- `SLURM_MEM_PER_NODE`: shared per-node memory upper bound.
- `SLURM_GPUS_PER_TASK`: requested GPUs per task.
- `SLURM_GPUS_ON_NODE`: GPUs allocated to the batch step on the node.
Slurm CPU confinement requires site configuration such as task affinity or
`task/cgroup` with core constraints. Memory requests are not strictly enforced
unless the site enables an enforcement mechanism. Use affinity and cgroup
observations as enforcement evidence; do not trust visible node inventory.
For other schedulers, use their documented allocation API or variables and the
same rule: allocation metadata and kernel enforcement are different facts.
Do not guess from generic environment names.
## Containers and OCI
Docker containers have no CPU or memory limit by default. When configured,
Docker maps CPU and memory flags to cgroup controls. OCI runtime configuration
also defines CPU, memory, and device constraints.
Inside a container:
- host inventory may remain visible;
- a CPU quota can be smaller than the visible CPU set;
- a cpuset can be smaller than the quota's apparent capacity;
- cgroup memory can be smaller than host RAM;
- GPU management tools can see a different set from the application runtime;
and
- mounted-volume capacity can differ from writable quota.
Always report observed cgroup controls and uncertainty. A container marker
without a finite cgroup value does not imply a finite limit.
## Disk
Capacity, free blocks, user-writable blocks, path permission, filesystem quota,
and actual ability to complete a write are different:
- capacity is the filesystem's total size;
- POSIX `f_bfree` includes blocks reserved from an unprivileged user;
- `shutil.disk_usage().free` uses `f_bavail` on POSIX, so the detector reads
`statvfs` to keep filesystem free and user-available values separate;
- POSIX `f_bavail` estimates blocks available to the current user;
- `os.access(..., os.W_OK)` is a non-writing permission check, not proof that a
future write will succeed;
- project/user quotas and remote storage policies can be stricter than block
counts.
The detector does not create a probe file. It redacts the absolute working path
and labels the scope as the working filesystem.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |