Skip to content

ahstn/oceans-llm / .agents/skills/verify-oceans-admin

verify-oceans-admin

Verified

Verify the Oceans LLM admin control plane and backend gateway against the real local stack, seeded demo data, and bounded live providers when required. Use for user-path checks of sign-in, models, API keys, observability, usage costs, spend controls, batches, agent sessions, request logs, Anthropic Messages thinking and effort compatibility, MCP registry and tool access, OpenRouter routing, guardrails, and Skills storage.

FileKindSize
SKILL.mdskill md29.0 KB
examples/mcp-candidates.jsonconfig3.0 KB
features/README.mdmarkdown10.0 KB
features/account-onboarding.mdmarkdown4.3 KB
features/agent-harnesses.mdmarkdown4.3 KB
features/agent-plugins.mdmarkdown1005 B
features/agent-sessions.mdmarkdown3.9 KB
features/anthropic-messages.mdmarkdown20.7 KB
features/api-keys.mdmarkdown6.5 KB
features/attribution.mdmarkdown13.9 KB
features/audit-log.mdmarkdown4.9 KB
features/backend-gateway.mdmarkdown5.2 KB
features/batch-requests.mdmarkdown4.2 KB
features/design-system.mdmarkdown4.6 KB
features/guardrails.mdmarkdown7.0 KB
features/harness-integration.mdmarkdown7.8 KB
features/identity.mdmarkdown4.2 KB
features/image-generation.mdmarkdown7.3 KB
features/leaderboard.mdmarkdown3.3 KB
features/live-llm-requests.mdmarkdown12.6 KB
features/mcp-client-oauth.mdmarkdown9.9 KB
features/mcp-ownership.mdmarkdown7.4 KB
features/mcp.mdmarkdown19.6 KB
features/models.mdmarkdown8.2 KB
features/password-sign-in.mdmarkdown8.3 KB
features/profile.mdmarkdown2.5 KB
features/rate-limits.mdmarkdown6.0 KB
features/request-logs.mdmarkdown8.6 KB
features/review-agent.mdmarkdown3.7 KB
features/routing.jsonconfig33.6 KB
features/service-tiers.mdmarkdown6.3 KB
features/skills.mdmarkdown9.8 KB
features/spend-controls.mdmarkdown7.3 KB
features/usage-costs.mdmarkdown3.7 KB
features/workspace-connections.mdmarkdown6.1 KB
scripts/check-messages.mjsexecutablescript2.4 KB
scripts/check-messages.test.mjsscript3.0 KB
scripts/control-oceans-adminexecutablescript25.3 KB
scripts/control-oceans-admin.test.mjsscript7.6 KB
scripts/drive-backend-gateway.mjsexecutablescript12.5 KB
scripts/drive-harness-integration.mjsscript6.6 KB
scripts/drive-mcp.mjsscript13.0 KB
scripts/drive-models.mjsscript12.9 KB
scripts/drive-observability.mjsscript16.8 KB
scripts/drive-profile.mjsscript8.2 KB
scripts/drive-service-tiers.mjsexecutablescript20.1 KB
scripts/drive-skills.mjsscript27.2 KB
scripts/mcp-browser.mjsscript26.5 KB
scripts/mcp-canary.mjsscript18.8 KB
scripts/mcp-candidates.mjsscript3.5 KB
scripts/mcp-verification.test.mjsscript10.2 KB
scripts/messages-regressions.mjsscript15.8 KB
scripts/plan-verification.mjsscript5.2 KB
scripts/plan-verification.test.mjsscript6.1 KB
scripts/proof-cli.mjsscript11.1 KB
scripts/proof.mjsscript9.2 KB
scripts/proof.test.mjsscript10.4 KB

scripts/drive-skills.mjs

View on GitHub
#!/usr/bin/env node

import assert from "node:assert/strict";
import { createHash, randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { createRequire } from "node:module";
import { fileURLToPath } from "node:url";
import { importBundledSkills } from "../../../../scripts/import-bundled-skills.mjs";
import { createProofRecorder } from "./proof.mjs";

const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../../..");
const requireFromAdminUi = createRequire(path.join(repoRoot, "crates/admin-ui/web/package.json"));
const { chromium, request } = requireFromAdminUi("playwright");
const { expect: baseExpect } = requireFromAdminUi("playwright/test");
const expect = baseExpect.configure({ timeout: 30_000 });
const baseURL = requiredEnv("OCEANS_VERIFY_BASE_URL");
const evidenceDir = requiredEnv("OCEANS_VERIFY_EVIDENCE_DIR");
requiredEnv("OCEANS_VERIFY_GATEWAY_VERSION");
const adminEmail = requiredEnv("OCEANS_VERIFY_ADMIN_EMAIL");
const adminPassword = requiredEnv("OCEANS_VERIFY_ADMIN_PASSWORD");
const serviceAccountKey = requiredEnv("OCEANS_VERIFY_SKILLS_SERVICE_ACCOUNT_KEY");
const suffix = randomUUID().replaceAll("-", "").slice(0, 12);
const fixturePath = (version) =>
  path.join(repoRoot, `crates/admin-ui/web/e2e/fixtures/skills/review-code-v${version}.zip`);
const actions = [];
const createdUsers = [];
const details = {
  entryUrl: `${baseURL}/admin/skills`,
  storage: {
    endpoint: requiredEnv("OCEANS_VERIFY_SKILLS_ENDPOINT"),
    bucket: requiredEnv("OCEANS_VERIFY_SKILLS_BUCKET"),
    prefix: requiredEnv("OCEANS_VERIFY_SKILLS_PREFIX"),
  },
  actions,
  cleanup: {
    usersDeactivated: [],
    usersRetainedForDatabaseCleanup: [],
    skills: "Stored in the run-local database; control-oceans-admin cleanup removes that database.",
    objects: "Run prefix is removed by control-oceans-admin cleanup.",
  },
};
// Stack cleanup adds a storage-cleanup check to this proof after teardown.
const proof = createProofRecorder({ evidenceDir, file: "skills-proof.json", featureIds: ["skills"] });
let browser;
let adminContext;
let currentPage;
let failure;

try {
  browser = await chromium.launch({ headless: true });
  adminContext = await browser.newContext({ viewport: { width: 1440, height: 1000 } });
  const adminPage = await adminContext.newPage();
  currentPage = adminPage;
  await adminPage.goto(details.entryUrl, { waitUntil: "domcontentloaded" });
  await expect(adminPage.getByRole("heading", { name: "Sign in", exact: true })).toBeVisible();
  await capture(adminPage, "01-skills-login");
  await fillSignIn(adminPage, adminEmail, adminPassword);
  await expect(adminPage.getByRole("heading", { name: "Agent Skills", exact: true })).toBeVisible();
  actions.push({
    action: "open protected Skills page and sign in as platform admin",
    result: "Agent Skills catalog visible",
  });
  proof.pass("protected-route-sign-in");

  details.bundled = await verifyBundledSkills(adminPage);
  proof.pass("bundled-import-idempotent", { observed: details.bundled.skills.length });

  const owner = await createUser(adminContext.request, "owner");
  const peer = await createUser(adminContext.request, "reader");
  const ownerContext = await browser.newContext({ viewport: { width: 1440, height: 1000 } });
  const ownerPage = await ownerContext.newPage();
  currentPage = ownerPage;
  await signIn(ownerPage, owner);
  await ownerPage.getByRole("link", { name: "Agent Skills", exact: true }).first().click();
  await expect(ownerPage.getByRole("heading", { name: "Agent Skills", exact: true })).toBeVisible();
  details.catalog = { before: await searchCatalog(ownerPage, owner.namespace) };
  assert.equal(details.catalog.before.apiCount, 0, "New namespace starts with an empty catalog");
  await capture(ownerPage, "02-skills-before");
  actions.push({
    action: "regular user follows Agent Skills sidebar and searches own namespace",
    result: "Empty catalog before upload",
  });
  proof.pass("empty-catalog-before-upload");

  const initial = await uploadFirstThroughUi(ownerPage, owner);
  const saved = await verifyOwnerVersions(ownerPage, initial);
  details.owner = { userId: owner.id, namespace: owner.namespace, skillId: saved.skill.id };
  details.versions = saved.versions.map(({ version, sha256 }) => ({ version, sha256 }));
  details.defaultVersion = saved.skill.default_version;
  proof.pass("owner-versions-immutable", { observed: details.versions.length });

  const peerContext = await browser.newContext({ viewport: { width: 1440, height: 1000 } });
  const peerPage = await peerContext.newPage();
  currentPage = peerPage;
  await signIn(peerPage, peer);
  const duplicateNamespace = await peerPage.request.post(`${baseURL}/api/v1/skills/namespace`, {
    data: { handle: owner.namespace },
  });
  assert.equal(
    duplicateNamespace.status(),
    409,
    "A namespace already owned by another user cannot be claimed",
  );
  assert.equal(
    await json(peerPage.request, "/api/v1/skills/namespace"),
    null,
    "Failed namespace claim leaves the user without a namespace",
  );
  proof.pass("namespace-claim-conflict", { observed: duplicateNamespace.status() });
  const duplicate = await uploadFirstThroughUi(peerPage, peer);
  assert.notEqual(duplicate.skill.id, initial.skill.id);
  assert.equal(duplicate.skill.name, initial.skill.name);
  assert.equal(duplicate.skill.owner_user_id, peer.id);
  details.peer = { userId: peer.id, namespace: peer.namespace, skillId: duplicate.skill.id };
  actions.push({
    action: "second user uploads the same skill name",
    result: "Separate owner namespace and skill ID",
  });
  proof.pass("duplicate-name-separate-owner");

  await verifySharedAccess(peerPage, adminContext.request, saved, owner);
  proof.pass("shared-read-owner-only-write");
  details.authentication = await verifyAuthentication(saved.skill.id);
  proof.pass("caller-authentication");
  await verifyNamespaceRules(ownerContext.request, peerContext.request, owner, peer);
  proof.pass("namespace-rules");
  currentPage = ownerPage;
  await ownerPage.goto(`${baseURL}/admin/skills`, {
    waitUntil: "domcontentloaded",
  });
  details.catalog.search = await verifyCatalogSearch(ownerPage, saved.skill, duplicate.skill);
  details.catalog.after = details.catalog.search.owner;
  assert.deepEqual(details.catalog.after.skillIds, [saved.skill.id]);
  await expectCatalogSkill(ownerPage, saved.skill);
  await capture(ownerPage, "07-skills-catalog-after");
  proof.pass("catalog-search", { observed: details.catalog.after.skillIds.length });
} catch (error) {
  failure = error;
  // A failed sign-in can still contain a password. Capture only the authenticated Skills surface.
  if (currentPage && /\/admin\/skills(?:[/?]|$)/.test(currentPage.url())) {
    await capture(currentPage, "99-skills-failure").catch(() => {});
  }
} finally {
  let cleanupFailed = false;
  for (const user of createdUsers) {
    // The last ordinary admin is protected by the normal lifecycle. The private database
    // is removed by stack cleanup; its temporary API key was already revoked.
    if (user.globalRole === "platform_admin") {
      details.cleanup.usersRetainedForDatabaseCleanup.push(user.id);
      continue;
    }
    try {
      const response = await adminContext.request.post(
        `${baseURL}/api/v1/admin/identity/users/${user.id}/deactivate`,
      );
      assert.equal(response.status(), 200, "Deactivate temporary verification user");
      details.cleanup.usersDeactivated.push(user.id);
    } catch (error) {
      cleanupFailed = true;
      failure ??= error;
    }
  }
  proof.add({ id: "user-cleanup", status: cleanupFailed ? "fail" : "pass", observed: details.cleanup.usersDeactivated.length });
  await proof.write({ details, error: failure && sanitizedFailure(failure) });
  await browser?.close();
}
if (failure) throw sanitizedFailure(failure);
console.log(
  "skills proof passed: browser upload, immutable versions, shared reads, owner-only changes, and archive digests matched the production API",
);
console.log(`evidence: ${evidenceDir}`);

async function verifyBundledSkills(page) {
  const demoAdmin = await createUser(page.request, "bundled-admin", "platform_admin");
  const created = await json(page.request, "/api/v1/admin/api-keys", {
    method: "POST",
    data: {
      name: `Bundled Skills Verification ${suffix}`,
      owner_kind: "user",
      owner_user_id: demoAdmin.id,
      owner_team_id: null,
      owner_service_account_id: null,
      model_grant_mode: "all",
      model_keys: [],
    },
  });
  let imported;
  try {
    const options = {
      url: baseURL,
      apiKey: created.data.raw_key,
      namespace: "demo-admin",
    };
    imported = await importBundledSkills(options);
    const repeated = await importBundledSkills(options);
    assert(repeated.skills.every((skill) => skill.unchanged), "Repeated local import is unchanged");
    assert.deepEqual(
      repeated.skills.map(({ id, version }) => ({ id, version })),
      imported.skills.map(({ id, version }) => ({ id, version })),
      "Repeated import preserves version identities",
    );
  } finally {
    await json(page.request, `/api/v1/admin/api-keys/${created.data.api_key.id}/revoke`, {
      method: "POST",
    });
  }
  const bundled = imported.skills.find((skill) => skill.name === "grill-me");
  assert(bundled, "Curated grill-me is imported");
  const detail = await json(page.request, `/api/v1/skills/${bundled.id}`);
  const version = await json(page.request, `/api/v1/skills/${bundled.id}/versions/${bundled.version}`);
  assert.equal(detail.skill.owner_user_id, demoAdmin.id, "Demo admin owns local import");
  assert.equal(detail.versions.length, 1, "Repeated import does not append an identical version");
  assert.equal(version.manifest.metadata.author, "Matt Pocock");
  assert.equal(version.manifest.license, "MIT");
  assert.equal(version.manifest.metadata.version, undefined, "No upstream version is invented");
  await page.goto(`${baseURL}/admin/skills/${bundled.id}`, { waitUntil: "domcontentloaded" });
  await expect(page.getByRole("heading", { name: "demo-admin/grill-me", exact: true })).toBeVisible();
  await expect(page.getByText("Author", { exact: true }).locator("..")).toContainText("Matt Pocock");
  await expect(page.getByRole("link", { name: "View on GitHub", exact: true })).toHaveAttribute(
    "href", version.manifest.metadata.github,
  );
  await expect(page.getByText("Contents", { exact: true }).locator("..")).toHaveText("Contents2 files");
  await expect(page.getByText("Uploaded", { exact: true }).locator("..")).toHaveText(
    /^Uploaded\d{1,2} [A-Z][a-z]{2} \d{4}, \d{2}:\d{2} UTC$/,
  );
  await expect(page.getByText("SHA-256", { exact: true })).toHaveCount(0);
  await expect(page.getByText("Upstream version", { exact: true })).toHaveCount(0);
  // The detail page can render before hydration attaches the file button handler.
  // Retry this read-only selection until the requested preview is visible.
  await expect(async () => {
    await page.getByRole("navigation", { name: "Skill files" })
      .getByRole("button", { name: "LICENSE", exact: true }).click();
    await expect(page.getByTestId("skill-file-text"), "Bundled license preview").toContainText(
      "Copyright (c) 2026 Matt Pocock", { timeout: 1_000 },
    );
  }).toPass({ timeout: 20_000, intervals: [250, 500, 1_000] });
  await page.getByRole("navigation", { name: "Skill files" })
    .getByRole("button", { name: "SKILL.md", exact: true }).click();
  await expect(page.getByTestId("skill-instructions")).toContainText("Interview me relentlessly");
  await capture(page, "01b-skills-bundled");
  actions.push({ action: "import curated skill as demo admin twice", result: "Attribution and license visible; unchanged second import; temporary key revoked" });
  return { ...imported, repeatedImportUnchanged: true, temporaryKeyRevoked: true };
}

async function createUser(adminApi, label, globalRole = "user") {
  const user = {
    name: `Skills Verification ${label}`,
    globalRole,
    email: `skills-${label}-${suffix}@example.com`,
    password: `Skills-${randomUUID()}-9`,
    namespace: `verify-${label}-${suffix}`,
  };
  const created = await json(adminApi, "/api/v1/admin/identity/users", {
    method: "POST",
    data: {
      name: user.name,
      email: user.email,
      auth_mode: "password",
      global_role: globalRole,
      tags: [],
    },
  });
  assert.equal(created.data.kind, "password_invite");
  user.id = created.data.user.id;
  createdUsers.push(user);
  const invite = new URL(created.data.invite_url, baseURL).pathname.split("/").findLast(Boolean);
  assert(invite, "Production invitation response must include a token");
  const anonymous = await request.newContext();
  try {
    const response = await anonymous.post(`${baseURL}/api/v1/auth/invitations/${invite}/password`, {
      data: { password: user.password },
    });
    assert.equal(response.status(), 200, "Complete temporary user password invitation");
  } finally {
    await anonymous.dispose();
  }
  return user;
}

async function signIn(page, user) {
  await page.goto(`${baseURL}/admin/login?redirect=/skills`, { waitUntil: "domcontentloaded" });
  await fillSignIn(page, user.email, user.password);
  await expect(page.getByRole("heading", { name: "Agent Skills", exact: true })).toBeVisible();
}

async function fillSignIn(page, email, password) {
  const signIn = page.getByRole("button", { name: "Sign in", exact: true });
  await expect(signIn).toBeEnabled({ timeout: 60_000 });
  await page.getByLabel("Email", { exact: true }).fill(email);
  await page.getByLabel("Password", { exact: true }).fill(password);
  await signIn.click();
}

async function expectCatalogSkill(page, skill) {
  const row = page.getByRole("table").getByRole("row")
    .filter({ has: page.getByRole("cell", { name: skill.namespace, exact: true }) })
    .filter({ has: page.getByRole("link", { name: skill.name, exact: true }) });
  const link = row.getByRole("link", { name: skill.name, exact: true });
  await expect(link).toBeVisible();
  await expect(link).toHaveAttribute("href", `/admin/skills/${skill.id}`);
}

async function searchCatalog(page, query) {
  const input = page.getByRole("textbox", { name: "Search skills", exact: true });
  // SSR can expose this controlled input before hydration attaches its change handler.
  // Retrying this read-only search proves that typing updates the URL without a submit button.
  await expect(async () => {
    if (query) await input.fill("");
    await input.fill(query);
    await expect(input).toHaveValue(query, { timeout: 1_000 });
    await expect(page).toHaveURL(
      (url) => (url.searchParams.get("q") ?? "") === query
        && Number(url.searchParams.get("offset") ?? 0) === 0,
      { timeout: 5_000 },
    );
    await expect(input).toHaveValue(query, { timeout: 1_000 });
  }).toPass({ timeout: 20_000, intervals: [250, 500, 1_000] });

  const items = await json(
    page.request,
    `/api/v1/skills?q=${encodeURIComponent(query)}&limit=50&offset=0`,
  );
  let uiCount = 0;
  if (items.length === 0) {
    await expect(page.getByText("No skills found", { exact: true })).toBeVisible();
    await expect(page.getByRole("table")).toHaveCount(0);
  } else {
    for (const skill of items) {
      await expectCatalogSkill(page, skill);
    }
    const rows = page.getByRole("table").getByRole("row");
    await expect(rows).toHaveCount(items.length + 1);
    uiCount = (await rows.count()) - 1;
    assert.equal(uiCount, items.length, "Filtered UI catalog matches the production API");
  }
  return {
    query,
    url: page.url(),
    uiCount,
    apiCount: items.length,
    skillIds: items.map((skill) => skill.id),
  };
}

async function verifyCatalogSearch(page, ownerSkill, peerSkill) {
  await expect(page.getByRole("button", { name: "Filter", exact: true })).toHaveCount(0);
  const expectedIds = new Set([ownerSkill.id, peerSkill.id]);
  const name = await searchCatalog(page, ownerSkill.name);
  assert.deepEqual(new Set(name.skillIds), expectedIds, "Name search finds both owners");
  const description = await searchCatalog(page, "repeatable checklist");
  assert.deepEqual(new Set(description.skillIds), expectedIds, "Description search finds both skills");
  const missing = await searchCatalog(page, `missing-${suffix}`);
  assert.equal(missing.apiCount, 0, "An unmatched search shows the empty state");
  const cleared = await searchCatalog(page, "");
  for (const id of [...expectedIds, ...details.bundled.skills.map((skill) => skill.id)]) {
    assert(cleared.skillIds.includes(id), "Clearing search restores each uploaded skill");
  }
  const owner = await searchCatalog(page, ownerSkill.namespace);
  assert.deepEqual(owner.skillIds, [ownerSkill.id], "Owner search keeps duplicate names distinct");
  actions.push({
    action: "search by name, description, and owner; check no match and clear",
    result: "Search updates without submit; row owners and destinations match the filtered API",
  });
  return { name, description, missing, cleared, owner };
}

async function uploadFirstThroughUi(page, owner) {
  const namespace = page.getByRole("dialog", { name: "Choose your skill namespace", exact: true });
  // Sign-in returns once the SSR heading is visible, so this click can land before
  // hydration attaches the handler that opens the dialog. Retry until it opens.
  await expect(async () => {
    if (!(await namespace.isVisible())) {
      await page.getByRole("button", { name: "Upload skill", exact: true }).click();
    }
    await expect(namespace).toBeVisible({ timeout: 2_000 });
  }).toPass({ timeout: 20_000, intervals: [250, 500, 1_000] });
  await namespace.getByLabel("Namespace", { exact: true }).fill(owner.namespace);
  await namespace.getByRole("button", { name: "Claim namespace", exact: true }).click();
  await uploadDialog(page, "Upload a skill", 1);
  await expect(
    page.getByRole("heading", { name: `${owner.namespace}/review-code`, exact: true }),
  ).toBeVisible();
  await expect(page.getByTestId("skill-instructions")).toContainText("checklist version 1");
  const detail = await json(page.request, `/api/v1/skills/by-name/${owner.namespace}/review-code`);
  assert.equal(detail.skill.owner_user_id, owner.id);
  assert.equal(detail.skill.default_version, 1);
  assert.equal(detail.skill.latest_version, 1);
  assert.equal(detail.versions.length, 1);
  assert.match(detail.versions[0].sha256, /^[a-f0-9]{64}$/);
  actions.push({
    action: "claim immutable namespace and upload ZIP through UI",
    result: `${owner.namespace}/review-code version 1`,
  });
  return detail;
}

async function uploadDialog(page, name, version) {
  const dialog = page.getByRole("dialog", { name, exact: true });
  await dialog.getByLabel("ZIP archive").setInputFiles(fixturePath(version));
  await dialog.getByRole("button", { name: "Upload skill", exact: true }).click();
}

async function verifyOwnerVersions(page, initial) {
  const skillPath = `/api/v1/skills/${initial.skill.id}`;
  await page
    .getByRole("navigation", { name: "Skill files" })
    .getByRole("button", { name: "references/checklist.md", exact: true })
    .click();
  await expect(page.getByTestId("skill-file-text")).toContainText("Checklist version 1");
  const file = await json(
    page.request,
    `${skillPath}/versions/1/files?path=references%2Fchecklist.md`,
  );
  const rendered = await page.getByTestId("skill-file-text").textContent();
  assert.equal(rendered, file.content, "Rendered file preview matches the production API");
  await capture(page, "03-skills-file-preview");

  await page.getByRole("button", { name: "Upload new version", exact: true }).click();
  await uploadDialog(page, "Upload a new version", 2);
  await expect(page.getByTestId("skill-instructions")).toContainText("checklist version 2");
  const appended = await json(page.request, skillPath);
  assert.equal(appended.skill.latest_version, 2);
  assert.equal(appended.skill.default_version, 1, "Appending a version must preserve the default");
  assert.equal(appended.versions.length, 2);
  await expect(page.getByText("Status", { exact: true })).toHaveCount(0);
  await page.getByRole("combobox", { name: "Version", exact: true }).click();
  await expect(page.getByRole("option", { name: "Version 1 (default)", exact: true })).toBeVisible();
  await expect(page.getByRole("option", { name: "Version 2 (latest)", exact: true })).toBeVisible();
  await page.keyboard.press("Escape");
  await capture(page, "04-skills-new-version");

  await page.getByRole("button", { name: "Set as default", exact: true }).click();
  await expect(page.getByText("Default version updated", { exact: true })).toBeVisible();
  await expect(page.getByRole("button", { name: "Set as default", exact: true })).toBeDisabled();
  const saved = await json(page.request, skillPath);
  assert.equal(saved.skill.default_version, 2);
  const downloadEvent = page.waitForEvent("download");
  await page.getByRole("link", { name: "Download ZIP", exact: true }).click();
  const download = await downloadEvent;
  assert.equal(await download.failure(), null, "Browser ZIP download completes");
  const digest = createHash("sha256")
    .update(await fs.readFile(await download.path()))
    .digest("hex");
  assert.equal(digest, saved.versions.find((item) => item.version === 2).sha256);
  await capture(page, "05-skills-owner-default");

  const old = await json(page.request, `${skillPath}/versions/1`);
  assert(old.instructions.includes("checklist version 1"), "Old instructions remain unchanged");
  const oldArchive = await page.request.get(`${baseURL}${skillPath}/versions/1/archive`);
  assert.equal(oldArchive.status(), 200);
  assert.equal(
    createHash("sha256")
      .update(await oldArchive.body())
      .digest("hex"),
    initial.versions[0].sha256,
  );
  actions.push({
    action: "upload version 2, change default, and download from the browser",
    result:
      "API saved default 2; downloaded digest matches; version 1 content and digest remain unchanged",
  });
  return saved;
}

async function verifySharedAccess(peerPage, adminApi, saved, owner) {
  const skillPath = `/api/v1/skills/${saved.skill.id}`;
  await peerPage.goto(`${baseURL}/admin/skills/${saved.skill.id}`, {
    waitUntil: "domcontentloaded",
  });
  await expect(
    peerPage.getByRole("heading", { name: `${owner.namespace}/review-code`, exact: true }),
  ).toBeVisible();
  await expect(peerPage.getByTestId("skill-instructions")).toContainText("checklist version 2");
  await expect(
    peerPage.getByRole("button", { name: "Upload new version", exact: true }),
  ).toHaveCount(0);
  await expect(peerPage.getByRole("button", { name: "Set as default", exact: true })).toHaveCount(
    0,
  );
  await capture(peerPage, "06-skills-other-owner");
  for (const api of [peerPage.request, adminApi]) {
    const read = await json(api, skillPath);
    assert.equal(read.skill.default_version, 2);
    const append = await api.post(`${baseURL}${skillPath}/versions`, {
      headers: { "content-type": "application/zip" },
      data: await fs.readFile(fixturePath(2)),
    });
    assert.equal(
      append.status(),
      403,
      "Only owner may append, including when caller is platform admin",
    );
    const choose = await api.put(`${baseURL}${skillPath}/default-version`, {
      data: { version: 1 },
    });
    assert.equal(choose.status(), 403, "Only owner may change default");
  }
  actions.push({
    action: "other user views skill; other user and platform admin attempt updates",
    result: "Shared read succeeds, owner controls hidden, both write paths return 403",
  });
}

async function verifyAuthentication(skillId) {
  const anonymous = await request.newContext();
  const checks = {
    anonymous: "passed",
    serviceAccount: "pending",
  };
  try {
    for (const endpoint of ["/api/v1/skills", `/api/v1/skills/${skillId}/versions/1/archive`]) {
      assert.equal(
        (await anonymous.get(`${baseURL}${endpoint}`)).status(),
        401,
        "Unauthenticated Skills request",
      );
    }
  } finally {
    await anonymous.dispose();
  }
  const serviceAccount = await request.newContext({
    extraHTTPHeaders: { authorization: `Bearer ${serviceAccountKey}` },
  });
  try {
    assert.equal(
      (await serviceAccount.get(`${baseURL}/api/v1/skills/${skillId}`)).status(),
      200,
      "Service account can read Skills",
    );
    const body = {
      headers: { "content-type": "application/zip" },
      data: await fs.readFile(fixturePath(2)),
    };
    assert.equal(
      (await serviceAccount.post(`${baseURL}/api/v1/skills`, body)).status(),
      403,
      "Service account cannot create a skill",
    );
    assert.equal(
      (await serviceAccount.post(`${baseURL}/api/v1/skills/${skillId}/versions`, body)).status(),
      403,
      "Service account cannot append a version",
    );
    assert.equal(
      (
        await serviceAccount.post(`${baseURL}/api/v1/skills/namespace`, {
          data: { handle: `verify-bot-${suffix}` },
        })
      ).status(),
      403,
      "Service account cannot claim a namespace",
    );
    assert.equal(
      (
        await serviceAccount.put(`${baseURL}/api/v1/skills/${skillId}/default-version`, {
          data: { version: 1 },
        })
      ).status(),
      403,
      "Service account cannot select default",
    );
    checks.serviceAccount = "passed";
  } finally {
    await serviceAccount.dispose();
  }
  actions.push({
    action: "check anonymous and service-account access",
    result: "Anonymous reads return 401; service-account reads succeed and all writes return 403",
  });
  return checks;
}

async function verifyNamespaceRules(ownerApi, peerApi, owner, peer) {
  const rename = await ownerApi.post(`${baseURL}/api/v1/skills/namespace`, {
    data: { handle: `renamed-${suffix}` },
  });
  assert.equal(rename.status(), 409, "Namespace cannot be changed after it is claimed");
  assert.equal((await json(ownerApi, "/api/v1/skills/namespace")).handle, owner.namespace);
  assert.equal((await json(peerApi, "/api/v1/skills/namespace")).handle, peer.namespace);
  actions.push({
    action: "attempt namespace change and namespace reuse",
    result: "Both return 409; original namespaces remain unchanged",
  });
}

async function json(api, route, options = {}) {
  const response = await api.fetch(`${baseURL}${route}`, options);
  assert.equal(response.status(), 200, `Production API ${route.split("?")[0]}`);
  return response.json();
}

async function capture(page, name) {
  await page.screenshot({ path: path.join(evidenceDir, `${name}.png`), fullPage: true });
  await fs.writeFile(
    path.join(evidenceDir, `${name}.aria.txt`),
    `${await page.locator("body").ariaSnapshot()}\n`,
  );
  proof.artifact(`${name}.png`);
  proof.artifact(`${name}.aria.txt`);
}

function requiredEnv(name) {
  const value = process.env[name];
  if (!value) throw new Error(`Missing required environment variable ${name}`);
  return value;
}

function sanitizedFailure(error) {
  // Playwright errors carry request headers in Call log and matcher metadata. Keep neither.
  let message = String(error?.message ?? "Skills verification failed")
    .replace(
      /^.*(?:authorization|proxy-authorization|cookie|set-cookie)\s*:.*$/gim,
      "[redacted header]",
    )
    .split(/\r?\n/)[0];
  for (const secret of [
    serviceAccountKey,
    adminPassword,
    ...createdUsers.map((user) => user.password),
  ]) {
    if (secret) message = message.replaceAll(secret, "[redacted]");
  }
  message = message.replace(/\/auth\/invitations\/[^/\s]+/g, "/auth/invitations/[redacted]");
  return new Error(`Skills verification failed: ${message.slice(0, 400)}`);
}
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall