Skip to content

ahstn/oceans-llm / .agents/skills/verify-oceans-admin

verify-oceans-admin

Verified

Verify the Oceans LLM admin control plane and backend gateway against the real local stack, seeded demo data, and bounded live providers when required. Use for user-path checks of sign-in, models, API keys, observability, usage costs, spend controls, batches, agent sessions, request logs, Anthropic Messages thinking and effort compatibility, MCP registry and tool access, OpenRouter routing, guardrails, and Skills storage.

FileKindSize
SKILL.mdskill md29.0 KB
examples/mcp-candidates.jsonconfig3.0 KB
features/README.mdmarkdown10.0 KB
features/account-onboarding.mdmarkdown4.3 KB
features/agent-harnesses.mdmarkdown4.3 KB
features/agent-plugins.mdmarkdown1005 B
features/agent-sessions.mdmarkdown3.9 KB
features/anthropic-messages.mdmarkdown20.7 KB
features/api-keys.mdmarkdown6.5 KB
features/attribution.mdmarkdown13.9 KB
features/audit-log.mdmarkdown4.9 KB
features/backend-gateway.mdmarkdown5.2 KB
features/batch-requests.mdmarkdown4.2 KB
features/design-system.mdmarkdown4.6 KB
features/guardrails.mdmarkdown7.0 KB
features/harness-integration.mdmarkdown7.8 KB
features/identity.mdmarkdown4.2 KB
features/image-generation.mdmarkdown7.3 KB
features/leaderboard.mdmarkdown3.3 KB
features/live-llm-requests.mdmarkdown12.6 KB
features/mcp-client-oauth.mdmarkdown9.9 KB
features/mcp-ownership.mdmarkdown7.4 KB
features/mcp.mdmarkdown19.6 KB
features/models.mdmarkdown8.2 KB
features/password-sign-in.mdmarkdown8.3 KB
features/profile.mdmarkdown2.5 KB
features/rate-limits.mdmarkdown6.0 KB
features/request-logs.mdmarkdown8.6 KB
features/review-agent.mdmarkdown3.7 KB
features/routing.jsonconfig33.6 KB
features/service-tiers.mdmarkdown6.3 KB
features/skills.mdmarkdown9.8 KB
features/spend-controls.mdmarkdown7.3 KB
features/usage-costs.mdmarkdown3.7 KB
features/workspace-connections.mdmarkdown6.1 KB
scripts/check-messages.mjsexecutablescript2.4 KB
scripts/check-messages.test.mjsscript3.0 KB
scripts/control-oceans-adminexecutablescript25.3 KB
scripts/control-oceans-admin.test.mjsscript7.6 KB
scripts/drive-backend-gateway.mjsexecutablescript12.5 KB
scripts/drive-harness-integration.mjsscript6.6 KB
scripts/drive-mcp.mjsscript13.0 KB
scripts/drive-models.mjsscript12.9 KB
scripts/drive-observability.mjsscript16.8 KB
scripts/drive-profile.mjsscript8.2 KB
scripts/drive-service-tiers.mjsexecutablescript20.1 KB
scripts/drive-skills.mjsscript27.2 KB
scripts/mcp-browser.mjsscript26.5 KB
scripts/mcp-canary.mjsscript18.8 KB
scripts/mcp-candidates.mjsscript3.5 KB
scripts/mcp-verification.test.mjsscript10.2 KB
scripts/messages-regressions.mjsscript15.8 KB
scripts/plan-verification.mjsscript5.2 KB
scripts/plan-verification.test.mjsscript6.1 KB
scripts/proof-cli.mjsscript11.1 KB
scripts/proof.mjsscript9.2 KB
scripts/proof.test.mjsscript10.4 KB

features/agent-harnesses.md

View on GitHub
# Agent Harnesses

Agent Harnesses ranks normalized caller harnesses by request count for a selected observability window and reports their summed input, output, and total tokens without adding token series to the request-count chart. The `Request activity` chart plots the top 5 harnesses in 12-hour buckets; the `Top Harnesses` ranking lists up to 30. With no data, both cards show `No harness data yet`.

## Sub-features

- `agent-harnesses.7d-api-parity` — The default 7-day desktop ranking matches the production harness-usage API.
- `agent-harnesses.tokens` — Each ranking row shows request, input-token, output-token, and total-token values, using `n/a` for null token aggregates.
- `agent-harnesses.mastra` — Seeded Mastra traffic (`demo-req-001`, `demo-req-010`) is rendered with the Mastra harness icon.
- `agent-harnesses.omp` — Seeded `omp/1.0.0` traffic (`demo-req-005`) is labeled Oh My Pi and rendered with the white `omp.sh` mark.
- `agent-harnesses.31d` — Selecting Last 31 days refreshes the chart and ranking from the 31-day API view.
- `agent-harnesses.states` — A range refresh shows `harness-chart-skeleton` and `harness-table-skeleton`; a failed refresh shows an error toast.
- `agent-harnesses.responsive` — Mobile cards replace the desktop table below the `md` breakpoint (route tests only; not driven).

## How to get to it (user POV)

- Open `/admin/observability/agent-harnesses` directly. An unauthenticated visit redirects to password sign-in and returns to Agent Harnesses after authentication.
- From another authenticated admin page, open `Observability` in the sidebar and select `Agent Harnesses`.

## Driving it with control-oceans-admin

Preconditions: launch a dedicated verification stack, require `doctor` to pass, and use the seeded `admin@local` platform administrator and refreshed local demo seed. Every signed-in user has the `agent_harnesses` page.

1. Run `control-oceans-admin drive observability`; after its Leaderboard checks, the driver follows the `Agent Harnesses` sidebar link.
2. Wait for the `Agent harnesses` heading, the `Request activity` and `Top Harnesses` cards, and `harness-usage-table`. The `Last 7 days` radio is checked. The desktop headers must include `Input tokens`, `Output tokens`, and `Total tokens`.
3. Read `/api/v1/admin/observability/harness-usage?range=7d` through the authenticated browser session. Every `harness-usage-table` row must match the API label, key, request count, and formatted nullable token values.
4. Find the API rows keyed `mastra` and `oh_my_pi`. The Mastra table row must contain `[data-agent-harness-icon="mastra"]`; the Oh My Pi row must contain `[data-agent-harness-icon="ohmypi"]` with a white path fill (`var(--color-white)`, computed `rgb(255, 255, 255)`) and no gradient.
5. Require the number of rendered chart areas to match `chart_harnesses`, then capture `04-agent-harnesses-7d.png` and `04-agent-harnesses-7d.aria.txt`.
6. Select the radio named `Last 31 days`. Wait until the rendered harness labels and request counts match `/api/v1/admin/observability/harness-usage?range=31d`, then capture `05-agent-harnesses-31d.png` and `05-agent-harnesses-31d.aria.txt`.
7. Require `observability-proof.json` to retain the API leaders, rendered headers and rows, window boundaries, series counts, Mastra and Oh My Pi icon counts, gateway version, and action log.

## Gotchas

- The route always loads 7 days first; do not infer a 31-day refresh from the selected radio alone. Wait for row values from the 31-day API response.
- `harness-usage-mobile-list` is present but hidden at the driver's desktop viewport. Use `harness-usage-table` for deterministic API comparison and verify responsive behavior in route tests.
- The chart plots at most 5 harnesses (`chart_harnesses`) while the ranking lists up to 30 `leaders`. Chart area count need not equal row count.
- The x-axis ticks once per UTC day, at the midnight bucket; with 31 days it shows an evenly spaced subset. The tooltip names the hovered bucket's start as `16 Oct 2026, 12:00 UTC`.
- Token sums are ranking-only. The chart remains request-count-only and its values must be compared with `series[].values[].request_count` semantics.
- Oh My Pi deliberately does not reuse the unrelated Pi icon. Its `omp.sh` mark must use a solid white fill, not the source gradient.
- This is a seeded read-only proof. It does not require a paid live LLM request.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall