Skip to content

ahstn/oceans-llm / .agents/skills/verify-oceans-admin

verify-oceans-admin

Verified

Verify the Oceans LLM admin control plane and backend gateway against the real local stack, seeded demo data, and bounded live providers when required. Use for user-path checks of sign-in, models, API keys, observability, usage costs, spend controls, batches, agent sessions, request logs, Anthropic Messages thinking and effort compatibility, MCP registry and tool access, OpenRouter routing, guardrails, and Skills storage.

FileKindSize
SKILL.mdskill md29.0 KB
examples/mcp-candidates.jsonconfig3.0 KB
features/README.mdmarkdown10.0 KB
features/account-onboarding.mdmarkdown4.3 KB
features/agent-harnesses.mdmarkdown4.3 KB
features/agent-plugins.mdmarkdown1005 B
features/agent-sessions.mdmarkdown3.9 KB
features/anthropic-messages.mdmarkdown20.7 KB
features/api-keys.mdmarkdown6.5 KB
features/attribution.mdmarkdown13.9 KB
features/audit-log.mdmarkdown4.9 KB
features/backend-gateway.mdmarkdown5.2 KB
features/batch-requests.mdmarkdown4.2 KB
features/design-system.mdmarkdown4.6 KB
features/guardrails.mdmarkdown7.0 KB
features/harness-integration.mdmarkdown7.8 KB
features/identity.mdmarkdown4.2 KB
features/image-generation.mdmarkdown7.3 KB
features/leaderboard.mdmarkdown3.3 KB
features/live-llm-requests.mdmarkdown12.6 KB
features/mcp-client-oauth.mdmarkdown9.9 KB
features/mcp-ownership.mdmarkdown7.4 KB
features/mcp.mdmarkdown19.6 KB
features/models.mdmarkdown8.2 KB
features/password-sign-in.mdmarkdown8.3 KB
features/profile.mdmarkdown2.5 KB
features/rate-limits.mdmarkdown6.0 KB
features/request-logs.mdmarkdown8.6 KB
features/review-agent.mdmarkdown3.7 KB
features/routing.jsonconfig33.6 KB
features/service-tiers.mdmarkdown6.3 KB
features/skills.mdmarkdown9.8 KB
features/spend-controls.mdmarkdown7.3 KB
features/usage-costs.mdmarkdown3.7 KB
features/workspace-connections.mdmarkdown6.1 KB
scripts/check-messages.mjsexecutablescript2.4 KB
scripts/check-messages.test.mjsscript3.0 KB
scripts/control-oceans-adminexecutablescript25.3 KB
scripts/control-oceans-admin.test.mjsscript7.6 KB
scripts/drive-backend-gateway.mjsexecutablescript12.5 KB
scripts/drive-harness-integration.mjsscript6.6 KB
scripts/drive-mcp.mjsscript13.0 KB
scripts/drive-models.mjsscript12.9 KB
scripts/drive-observability.mjsscript16.8 KB
scripts/drive-profile.mjsscript8.2 KB
scripts/drive-service-tiers.mjsexecutablescript20.1 KB
scripts/drive-skills.mjsscript27.2 KB
scripts/mcp-browser.mjsscript26.5 KB
scripts/mcp-canary.mjsscript18.8 KB
scripts/mcp-candidates.mjsscript3.5 KB
scripts/mcp-verification.test.mjsscript10.2 KB
scripts/messages-regressions.mjsscript15.8 KB
scripts/plan-verification.mjsscript5.2 KB
scripts/plan-verification.test.mjsscript6.1 KB
scripts/proof-cli.mjsscript11.1 KB
scripts/proof.mjsscript9.2 KB
scripts/proof.test.mjsscript10.4 KB

features/leaderboard.md

View on GitHub
# Leaderboard

The Leaderboard ranks users by priced or legacy-estimated spend for a selected observability window and shows each user's most-used model, most-used harness, request volume, and average tool counts. Requests without priced usage still count toward request volume. The `User costs over time` chart plots the top 5 users in 12-hour UTC buckets; the `Top Users` ranking lists up to 30 users. With no data, both cards show `No leaderboard data yet`.

## Sub-features

- `leaderboard.7d-api-parity` — The default 7-day desktop ranking matches the production leaderboard API.
- `leaderboard.model` — Each user shows the single most-used model key.
- `leaderboard.harness` — Each user shows the label of the most-used normalized agent harness.
- `leaderboard.31d` — Selecting Last 31 days refreshes the chart and ranking from the 31-day API view.
- `leaderboard.responsive` — Mobile cards replace the desktop table below the `md` breakpoint.

## How to get to it (user POV)

- Open `/admin/observability/leaderboard` directly. An unauthenticated visit redirects to password sign-in and returns to the Leaderboard after authentication.
- From another authenticated admin page, open `Budget & Spending` in the sidebar and select `Leaderboard`.

## Driving it with control-oceans-admin

Preconditions: launch a dedicated verification stack, require `doctor` to pass, and use the seeded `admin@local` platform administrator. Every role has the `leaderboard` page.

1. Run `control-oceans-admin drive observability`; the driver opens the protected Leaderboard route and signs in through the visible `Sign in` form.
2. Wait for the `Leaderboard` heading and `leaderboard-table`. The desktop headers must include `Most used model` and `Most used harness`.
3. Read `/api/v1/admin/observability/leaderboard?range=7d` through the authenticated browser session. Every `leaderboard-table` row must match the API rank, user, spend, model key, harness label, request count, and all four average tool counts.
4. Require the number of rendered chart areas to match `chart_users`, then capture `02-leaderboard-7d.png` and `02-leaderboard-7d.aria.txt`.
5. Select the radio named `Last 31 days` (visible text `31d`). Wait until every rendered row, including average tool counts, matches `/api/v1/admin/observability/leaderboard?range=31d`, then capture `03-leaderboard-31d.png` and `03-leaderboard-31d.aria.txt`.
6. Resize to 600×1000. Require `leaderboard-mobile-list` visible and `leaderboard-table` hidden, capture `03b-leaderboard-mobile`, then restore 1440×1000.
7. Require `observability-proof.json` to retain the API leaders, rendered headers and rows, window boundaries, series counts, gateway version, and action log.

## Gotchas

- The route always loads 7 days first; do not infer a 31-day refresh from the selected radio alone. Wait for row values from the 31-day API response.
- `leaderboard-mobile-list` is in the DOM but hidden (`md:hidden`) at 1440px. Use `leaderboard-table` for API comparison. The responsive step checks visibility at 600px, not DOM presence.
- The x-axis ticks once per UTC day, at the midnight bucket; with 31 days it shows an evenly spaced subset. The tooltip names the hovered bucket's start as `16 Oct 2026, 12:00 UTC`.
- The chart is spend-only. Most-used model and harness values belong to ranking rows and must not be inferred from chart labels.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall