Skip to content

ahstn/oceans-llm / .agents/skills/verify-oceans-admin

verify-oceans-admin

Verified

Verify the Oceans LLM admin control plane and backend gateway against the real local stack, seeded demo data, and bounded live providers when required. Use for user-path checks of sign-in, models, API keys, observability, usage costs, spend controls, batches, agent sessions, request logs, Anthropic Messages thinking and effort compatibility, MCP registry and tool access, OpenRouter routing, guardrails, and Skills storage.

FileKindSize
SKILL.mdskill md29.0 KB
examples/mcp-candidates.jsonconfig3.0 KB
features/README.mdmarkdown10.0 KB
features/account-onboarding.mdmarkdown4.3 KB
features/agent-harnesses.mdmarkdown4.3 KB
features/agent-plugins.mdmarkdown1005 B
features/agent-sessions.mdmarkdown3.9 KB
features/anthropic-messages.mdmarkdown20.7 KB
features/api-keys.mdmarkdown6.5 KB
features/attribution.mdmarkdown13.9 KB
features/audit-log.mdmarkdown4.9 KB
features/backend-gateway.mdmarkdown5.2 KB
features/batch-requests.mdmarkdown4.2 KB
features/design-system.mdmarkdown4.6 KB
features/guardrails.mdmarkdown7.0 KB
features/harness-integration.mdmarkdown7.8 KB
features/identity.mdmarkdown4.2 KB
features/image-generation.mdmarkdown7.3 KB
features/leaderboard.mdmarkdown3.3 KB
features/live-llm-requests.mdmarkdown12.6 KB
features/mcp-client-oauth.mdmarkdown9.9 KB
features/mcp-ownership.mdmarkdown7.4 KB
features/mcp.mdmarkdown19.6 KB
features/models.mdmarkdown8.2 KB
features/password-sign-in.mdmarkdown8.3 KB
features/profile.mdmarkdown2.5 KB
features/rate-limits.mdmarkdown6.0 KB
features/request-logs.mdmarkdown8.6 KB
features/review-agent.mdmarkdown3.7 KB
features/routing.jsonconfig33.6 KB
features/service-tiers.mdmarkdown6.3 KB
features/skills.mdmarkdown9.8 KB
features/spend-controls.mdmarkdown7.3 KB
features/usage-costs.mdmarkdown3.7 KB
features/workspace-connections.mdmarkdown6.1 KB
scripts/check-messages.mjsexecutablescript2.4 KB
scripts/check-messages.test.mjsscript3.0 KB
scripts/control-oceans-adminexecutablescript25.3 KB
scripts/control-oceans-admin.test.mjsscript7.6 KB
scripts/drive-backend-gateway.mjsexecutablescript12.5 KB
scripts/drive-harness-integration.mjsscript6.6 KB
scripts/drive-mcp.mjsscript13.0 KB
scripts/drive-models.mjsscript12.9 KB
scripts/drive-observability.mjsscript16.8 KB
scripts/drive-profile.mjsscript8.2 KB
scripts/drive-service-tiers.mjsexecutablescript20.1 KB
scripts/drive-skills.mjsscript27.2 KB
scripts/mcp-browser.mjsscript26.5 KB
scripts/mcp-canary.mjsscript18.8 KB
scripts/mcp-candidates.mjsscript3.5 KB
scripts/mcp-verification.test.mjsscript10.2 KB
scripts/messages-regressions.mjsscript15.8 KB
scripts/plan-verification.mjsscript5.2 KB
scripts/plan-verification.test.mjsscript6.1 KB
scripts/proof-cli.mjsscript11.1 KB
scripts/proof.mjsscript9.2 KB
scripts/proof.test.mjsscript10.4 KB

features/attribution.md

View on GitHub
# Attribution

Attribution is a read-only report that joins Git AI authorship notes with gateway traffic over a 7-, 30-, or 90-day window. It shows how many lines of each commit AI wrote, which merged pull requests those commits reached, and, in its tables, the agent requests and priced cost behind them. Platform administrators see every user and can narrow the report to one user or repository; everyone else sees only the commits they own or authored. The data arrives from `oceans hook` on developer machines, so the page also explains the hook install when it has nothing to show.

## Sub-features

- `attribution-kpis` shows the `Commits`, `Merged PRs`, and `AI lines` cards. Requests, cost, and models have no cards here; Usage Costs owns them.
- `attribution-window` switches the report between the last 7, 30, and 90 days through `?days=`.
- `attribution-repository-filter` narrows the report to one repository through the `Repository filter` combobox and `?repository=`.
- `attribution-user-filter` narrows a platform-admin report to one user: the `Filter the report to <name>` button in a `User activity` row sets `?user_id=`, and the `Clear user filter: <name>` button drops it.
- `attribution-pull-requests` lists merged pull requests in the `Merged pull requests` table, linked to the forge.
- `attribution-changes` lists commits in the `Recent changes` table, with a joined or unjoined session badge and `Show more` past 50 rows.
- `attribution-empty` shows the hook install steps when the window has no data, and `Clear filters` when only the filters exclude everything.
- `attribution-request-journal` joins a gateway request to the journal ID reported by the OMP response hook, even when its native provider session ID differs.

## How to get to it (user POV)

- Sign in and choose `Attribution` under `Observability`.
- Open `/admin/observability/attribution` directly. `?days=7|30|90`, `?repository=<host/owner/repo>`, and `?user_id=<uuid>` deep-link a filtered report.

## Driving it with control-oceans-admin

Preconditions:

- `control-oceans-admin doctor` passes.
- `mise run dev-stack` completed its local demo seed. Each run rewrites four commits at fresh relative times, in `github.com/oceans-demo/admin-console` and `github.com/oceans-demo/release-runbooks`:
  - `Gina Research Engineer` (`gina@research.local`) has an `admin-console` commit (about 55 minutes old) joined to her seeded Codex session, and a `release-runbooks` commit (about 150 seconds old) joined to another session that made 3 more requests after it.
  - `Alice Platform Lead` (`alice@platform.local`) has an `admin-console` commit (2 days old) whose Claude session is not joined, and a human-only commit (4 days old). Its Git author is not an Oceans user, but the note counts for Alice because her key sent it.
  - Two merged pull requests in `admin-console`: `#42` `Rename the observability route to sessions` by Gina, opened 6 hours before the seed and approved 3 hours later, and `#41` `Summarise release blockers on the dashboard` by Alice, merged without review. The `release-runbooks` commit has no pull request.
  - Two more that never merged: `#43` `Add a cost column to the harness table`, Gina's open draft, and `#40` `Try a denser pull request ledger`, which Alice closed without merging after two reviews.
- You are signed in as the seeded `admin@local` platform administrator. The page shares Usage Costs' `usage_costs` page, which comes from the `users` page set that platform administrators inherit.

- **Open.** Choose the `Attribution` link. The `Attribution` heading appears under the `Observability` section label, above an `Attribution sections` nav with `Impact`, `Delivery`, `Pull requests`, `Durability` and `My work`, unnumbered, `Impact` current. `Repositories` is off by default. The header ends with a `Setup` link that opens the setup guide in a new tab. With the seed, `AI share of merged code` reads `74%` (212 AI lines merged), `Cost per merged PR` reads `$5.11`, `Merged PRs` reads `2` with `4 commits`, and `Spend that shipped` reads `100%`. `Session outcomes` draws `All sessions` (3) into `Committed`, then `Merged PR` (2) and `Default branch` (1). The `AI lines unchanged after 30 days` counterweight shows `—` with a `Data` pill; reverts carry a `Proxy` pill.
- **Tabs.** `Delivery` shows five DORA tiles; `Deployment rework rate` carries a warning `Data` pill; the deploy, change-fail and recovery tiles carry a `≈` proxy icon beside their headings. Below them, the review and flow row reads `Review wait` from the seed's two reviews (3 hours on `#42`, a day on the closed `#40`), `Merged without review` `50%` (`#41`, 1 of 2), `Abandon rate` from `#40` (1 of 3 closed), and `Open pull requests` `1` with `1 draft` (`#43`). The open and closed pull requests never appear in the ledger or the merged counts. `Pull requests` lists `#42` and `#41`; Clicking anywhere on the `#42` row (or `Expand #42`) shows one commit, `4f2c9a1`, by Codex, with `Review wait: 3 h` and `1 review, 1 approving`; `#41` reads `Merged without review`. A pull request that reverts another shows `Reverts #N` (its evidence in the badge's title) and the original `Reverted by #N`; a title-only one shows `Possible revert`. The seed has no reverts; the gateway and service tests cover them. With `attribution.pages.repositories: true`, `Repositories` lists `oceans-demo/admin-console` and `oceans-demo/release-runbooks`, each with a `Data` durability pill. `Durability` shows `No survival data yet`. `My work` is the signed-in user's own report; `admin@local` has no seeded commits, so it shows `No attributed commits yet`.
- **Flags.** With `attribution.pages.delivery: false` in the gateway config and a restart, `Delivery` disappears from the nav. With every tab off the page reads `Every Attribution tab is turned off`. `/admin/observability/attribution/concepts` redirects to the page unless `attribution.pages.concepts` is on.
- **Agent Harnesses.** `What each harness ships` shows a tile per harness with cost per 100 AI lines and a `Data` steering pill, then `Harness by model` with `Cost per line`, `Shipped` and `Steering` views, above `Request activity`.
- **Window.** In the `Report window` group, press the `Last 7 days` toggle (visible text `7d`). The URL gains `days=7`, `Refresh` reads `Refreshing...`, and the controls are disabled until the report loads. Once it settles, `7d` is pressed and the seeded totals are unchanged, because every seeded commit is under 7 days old. Press `Last 30 days` to return.
- **Repository filter.** Open the `Repository filter` combobox and choose `github.com/oceans-demo/release-runbooks`. The URL gains `repository=`. On `Pull requests`, `Recent changes` shows only Gina's commit and `Merged pull requests` reads `No merged pull requests in this window.` Choose `All repositories` to reset.
- **User filter.** In the `User activity` row for `Gina Research Engineer`, press the `Filter the report to Gina Research Engineer` button (visible text is her name). The URL gains `user_id=`, the button gets `aria-pressed="true"` and the row `aria-current="true"`, a `Clear user filter: Gina Research Engineer` button appears next to the window toggle, and `Merged PRs` reads `1`. Press that button to drop the filter.
- **Tables.** In `Merged pull requests`, `oceans-demo/admin-console#42` (the `github.com/` host is left off) is a link to the pull request. In `Recent changes`, each row shows a 7-character SHA, with the agent tool underneath when the note names one. Gina's `admin-console` commit shows a success `1 joined` badge and links `#42`. Alice's `admin-console` commit shows a warning `1 unjoined` badge. Alice's human-only commit reads `Alice Platform Lead` as its author.
- **Unknown cost.** Alice's unjoined commit has no priced request, so its `Cost` reads `—`, never `$0.00`. Rows with unpriced or usage-missing requests also show a warning `N gaps` badge.
- **Empty states.** Open `/admin/observability/attribution?repository=github.com/example/none`. The page shows `No attribution matches these filters` and a `Clear filters` button, which drops both filters. A window with no data at all shows `No attributed commits yet`, the `oceans hook install --git-ai --claude` and `oceans hook install --git-ai --omp` commands, and a `Set up Git AI attribution` link to `https://oceans-llm.com/harnesses/git-ai-attribution/`.
- **API parity.** With the same session, fetch `GET /api/v1/admin/attribution/report?days=30`. `data.summary.merged_pull_request_count` must match `Merged PRs` on Impact and `data.summary.commit_count` its `N commits` hint, `data.summary.merged_ai_lines` the `AI share of merged code` hint, and `data.summary.median_pull_request_cost_usd_10000` `Cost per merged PR`. `data.totals.unmatched_author_commit_count` must be `0`. `data.users` must have one entry per `User activity` row, and the first `data.changes[].commit_sha` must start with the first `Recent changes` SHA.
- **Proof.** Capture the default 30-day view, the Gina-filtered or `release-runbooks` view, and the API totals.
- **Concept mockups.** With `attribution.pages.concepts: true`, open `/admin/observability/attribution/concepts`. The `Attribution` sidebar link stays active, the header reads `Observability · Concepts`, and an `Attribution concepts` nav shows eight tabs from `1 Impact` to `8 My work`. Every tab renders sample data, not the seed: the `Concept mockup on sample data` alert and a readiness badge (`Live`, `Derivable`, `Proxy` or a warning `Data` pill) on each card. Pressing `90d` adds `days=90` to the URL and changes the figures. On `4 Pull requests`, `Expand #…` opens a commit timeline and a per-commit table. No tab sends an attribution API request.

### Request-to-journal proof

Run this proof when the OMP extension, attribution ingestion or request-fact joins change. Use only the launched worktree stack, its own database and a temporary key with the same owner for inference and attribution. Never install a verification extension into the user's live OMP profile.

1. Build the CLI through `mise run //crates/oceans-cli:build`. Create an isolated OMP profile with the updated extension and an Oceans model pointed at the recorded gateway URL. Use an OMP version with `after_provider_response`, such as 18.20.2. Do not set `OCEANS_SESSION_ID` or inject a journal-valued native session header.
2. Send one small synthetic request with an output cap of 4096 tokens or less. Follow the live-request policy if it reaches a real provider. Capture only its gateway request ID, journal ID hash, status, model and usage counts; keep raw IDs and credentials out of shared artifacts.
3. Include a non-UUID `x-request-id` trace value on the canary request. Confirm the response keeps that trace value while `x-oceans-request-id` is a server UUID matching the stored request and usage ledger. An upstream provider request ID must not replace it. Flush the isolated queue with `oceans hook send --foreground`. Confirm the accepted `request_session` mapping stores the current context's journal hash under the inference key's owner. The detached request hook must deliver in-process without starting another sender process.
4. Send a valid Git AI note for that exact journal with a commit time after the request. Filter Attribution to the proof repository. The commit must show one joined session, that request once and its priced cost once. A different native provider ID must not prevent the join, and a parent journal must not receive a child's mapped request.
5. Repeat the offline arrival-order, stale/duplicate and cross-owner regression cases. For Codex, prove requests that share a native session but have distinct thread execution IDs join only the notes for their own threads. These cases must not use model, time or account similarity as identity evidence.
6. Open the filtered Attribution page and the matching Request Logs detail in the real browser. Capture both, with payloads masked, plus sanitized API totals. Revoke the temporary key, then stop only the stack and processes started for the proof.

## Gotchas

- Unlike Usage Costs, the window and filters live in the URL, so a reload keeps them. The gateway defaults to 7 days, but the page always sends `days` and defaults to 30.
- Non-platform-admin sessions get a self-only report. Their `User activity` rows have no filter button, and the gateway ignores `user_id` for them. An empty self-only report shows the hook install steps, not `Clear filters`.
- The seed uses times relative to its run, so a stack seeded days ago can push Alice's 4-day-old commit out of the 7-day window. Rerun the seed rather than adjusting the expected counts.
- Outside the seed, notes reach Oceans from `oceans hook` on a developer machine: `oceans login`, `oceans hook install --git-ai --claude` (or `--omp`), then an agent-made commit. The CLI queues entries and sends them in the background; `oceans hook send --foreground` flushes the queue, and `oceans hook queue` shows pending and quarantined entries.
- A note sent with a user's key always counts for that user. Author emails decide only for pull request commits and for notes from a service account key. They match a user's email, then the `additional_attribution_emails` of a config `users` entry. `Unmatched author` therefore shows only on a pull request, or a service account's commit, whose author email matched no user.
- Merged pull requests come only from `oceans hook prs`, which uses the local `gh`. There is no server-side GitHub token. A commit's `#N` links only when that PR is in the window's `Merged pull requests`.
- A session is joined only through an exact native ID, an evidenced Codex thread execution ID or an explicit request-to-journal mapping. A mapping takes priority over the provider's native session so a child request cannot also count for its parent's journal. Lines still count without a request match, but add no requests or cost.
- Cost sums priced requests only, so a table row with unpriced or usage-missing requests shows a `N gaps` badge.
- The report reads at most 1,000 commits, 1,000 pull requests and 1,000 hook-reported sessions. When it hits a read limit, it shows `This report is incomplete`; choose a shorter window or a filter.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall