ahstn/oceans-llm / .agents/skills/verify-oceans-admin
verify-oceans-admin
VerifiedVerify the Oceans LLM admin control plane and backend gateway against the real local stack, seeded demo data, and bounded live providers when required. Use for user-path checks of sign-in, models, API keys, observability, usage costs, spend controls, batches, agent sessions, request logs, Anthropic Messages thinking and effort compatibility, MCP registry and tool access, OpenRouter routing, guardrails, and Skills storage.
features/agent-harnesses.md
View on GitHub# Agent Harnesses
Agent Harnesses ranks normalized caller harnesses by request count for a selected observability window and reports their summed input, output, and total tokens without adding token series to the request-count chart. The `Request activity` chart plots the top 5 harnesses in 12-hour buckets; the `Top Harnesses` ranking lists up to 30. With no data, both cards show `No harness data yet`.
## Sub-features
- `agent-harnesses.7d-api-parity` — The default 7-day desktop ranking matches the production harness-usage API.
- `agent-harnesses.tokens` — Each ranking row shows request, input-token, output-token, and total-token values, using `n/a` for null token aggregates.
- `agent-harnesses.mastra` — Seeded Mastra traffic (`demo-req-001`, `demo-req-010`) is rendered with the Mastra harness icon.
- `agent-harnesses.omp` — Seeded `omp/1.0.0` traffic (`demo-req-005`) is labeled Oh My Pi and rendered with the white `omp.sh` mark.
- `agent-harnesses.31d` — Selecting Last 31 days refreshes the chart and ranking from the 31-day API view.
- `agent-harnesses.states` — A range refresh shows `harness-chart-skeleton` and `harness-table-skeleton`; a failed refresh shows an error toast.
- `agent-harnesses.responsive` — Mobile cards replace the desktop table below the `md` breakpoint (route tests only; not driven).
## How to get to it (user POV)
- Open `/admin/observability/agent-harnesses` directly. An unauthenticated visit redirects to password sign-in and returns to Agent Harnesses after authentication.
- From another authenticated admin page, open `Observability` in the sidebar and select `Agent Harnesses`.
## Driving it with control-oceans-admin
Preconditions: launch a dedicated verification stack, require `doctor` to pass, and use the seeded `admin@local` platform administrator and refreshed local demo seed. Every signed-in user has the `agent_harnesses` page.
1. Run `control-oceans-admin drive observability`; after its Leaderboard checks, the driver follows the `Agent Harnesses` sidebar link.
2. Wait for the `Agent harnesses` heading, the `Request activity` and `Top Harnesses` cards, and `harness-usage-table`. The `Last 7 days` radio is checked. The desktop headers must include `Input tokens`, `Output tokens`, and `Total tokens`.
3. Read `/api/v1/admin/observability/harness-usage?range=7d` through the authenticated browser session. Every `harness-usage-table` row must match the API label, key, request count, and formatted nullable token values.
4. Find the API rows keyed `mastra` and `oh_my_pi`. The Mastra table row must contain `[data-agent-harness-icon="mastra"]`; the Oh My Pi row must contain `[data-agent-harness-icon="ohmypi"]` with a white path fill (`var(--color-white)`, computed `rgb(255, 255, 255)`) and no gradient.
5. Require the number of rendered chart areas to match `chart_harnesses`, then capture `04-agent-harnesses-7d.png` and `04-agent-harnesses-7d.aria.txt`.
6. Select the radio named `Last 31 days`. Wait until the rendered harness labels and request counts match `/api/v1/admin/observability/harness-usage?range=31d`, then capture `05-agent-harnesses-31d.png` and `05-agent-harnesses-31d.aria.txt`.
7. Require `observability-proof.json` to retain the API leaders, rendered headers and rows, window boundaries, series counts, Mastra and Oh My Pi icon counts, gateway version, and action log.
## Gotchas
- The route always loads 7 days first; do not infer a 31-day refresh from the selected radio alone. Wait for row values from the 31-day API response.
- `harness-usage-mobile-list` is present but hidden at the driver's desktop viewport. Use `harness-usage-table` for deterministic API comparison and verify responsive behavior in route tests.
- The chart plots at most 5 harnesses (`chart_harnesses`) while the ranking lists up to 30 `leaders`. Chart area count need not equal row count.
- The x-axis ticks once per UTC day, at the midnight bucket; with 31 days it shows an evenly spaced subset. The tooltip names the hovered bucket's start as `16 Oct 2026, 12:00 UTC`.
- Token sums are ranking-only. The chart remains request-count-only and its values must be compared with `series[].values[].request_count` semantics.
- Oh My Pi deliberately does not reuse the unrelated Pi icon. Its `omp.sh` mark must use a solid white fill, not the source gradient.
- This is a seeded read-only proof. It does not require a paid live LLM request.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |