Skip to content

netresearch/matrix-skill / skills/matrix-announcement

matrix-announcement

Verified

Use when composing a Matrix announcement β€” skill release, version bump, weekly digest, breaking-change heads-up, postmortem, RFC, multi-skill pipeline summary, or any agent-authored room post longer than a single line. Trigger before any matrix-send call that produces structured content. Companion to matrix-communication.

evals/evals.json

View on GitHub
{
  "skill_name": "matrix-announcement",
  "evals": [
    {
      "id": 1,
      "eval_name": "type-tag-required",
      "prompt": "Announce the v3.12.0 release of jira-skill in #releases:example.com.",
      "expected_output": "Composes a message that begins with a single type tag (e.g. 'Release: jira-skill v3.12.0') and a leading glyph.",
      "files": [],
      "assertions": [
        "First line includes one of: Release / Patch / Heads-up / Digest / Postmortem / RFC / New skill",
        "First line includes at most one prefix glyph",
        "Does NOT use πŸš€ or πŸŽ‰",
        "Does NOT begin with 'We're excited to'"
      ]
    },
    {
      "id": 2,
      "eval_name": "lists-not-prose",
      "prompt": "Compose a release announcement listing four changes: ecosystem detection, version bumps, signed tags, narrative release notes.",
      "expected_output": "Renders the four items as a <ul>, not as a paragraph with 'and also...'.",
      "files": [],
      "assertions": [
        "Uses <ul> or <ol> with <li> for the four items",
        "Does NOT pack all four into a single paragraph",
        "Each list item is a complete bullet, not a fragment"
      ]
    },
    {
      "id": 3,
      "eval_name": "code-tag-discipline",
      "prompt": "Tell the room how to install the github-release-skill plugin and which command it blocks.",
      "expected_output": "Wraps every command, path, version string, and identifier in <code>.",
      "files": [],
      "assertions": [
        "/install-plugin invocation wrapped in <code>",
        "Blocked command (gh release create) wrapped in <code>",
        "Skill name (github-release-skill) wrapped in <code>",
        "Version strings wrapped in <code>"
      ]
    },
    {
      "id": 4,
      "eval_name": "no-raw-urls",
      "prompt": "Include a link to the repository at https://github.com/netresearch/matrix-skill in the message.",
      "expected_output": "Wraps the URL in <a href='...'> with destination-as-text (org/repo or similar), not 'click here'.",
      "files": [],
      "assertions": [
        "URL is in href, not in body",
        "Anchor text is the destination identity, not 'here' or 'click here'",
        "Plaintext body keeps the raw URL alongside the identifier"
      ]
    },
    {
      "id": 5,
      "eval_name": "render-png-for-tabular",
      "prompt": "Post a weekly digest covering 17 repos with old and new versions for each.",
      "expected_output": "Renders templates/weekly-digest.html to PNG via headless Chromium and posts m.image, instead of inlining a 17-row table in formatted_body.",
      "files": [],
      "assertions": [
        "Surfaces that >6-row tables don't render well in formatted_body",
        "Uses chromium --headless=new with --window-size=1200,1500",
        "Sends m.image with mxc:// URI",
        "Pairs the image with a plaintext body fallback or a follow-up m.text"
      ]
    },
    {
      "id": 6,
      "eval_name": "headsup-for-breaking",
      "prompt": "Announce that matrix-skill v2.0.0 drops Python 3.8.",
      "expected_output": "Uses the 'Heads-up' tag with ⚠️ glyph; states migration steps and timeline; does NOT use the 'Release' tag for this.",
      "files": [],
      "assertions": [
        "Type tag is Heads-up, not Release",
        "Glyph is ⚠️, not πŸ“¦ or πŸš€",
        "Includes Migration step",
        "Includes Timeline / sunset date"
      ]
    },
    {
      "id": 7,
      "eval_name": "m-notice-for-bots",
      "prompt": "Send the release announcement automatically from CI without human review.",
      "expected_output": "Sets msgtype to m.notice (not m.text) so other bots won't auto-reply. Uses the matrix-send-e2ee.py --notice flag (the transport's native support) instead of falling back to a raw API call.",
      "files": [],
      "assertions": [
        "Surfaces m.notice vs m.text distinction",
        "Recommends m.notice for the unattended/CI case",
        "Uses the --notice flag on matrix-send-e2ee.py / matrix-send.py rather than a raw curl PUT"
      ]
    },
    {
      "id": 8,
      "eval_name": "thread-long-content",
      "prompt": "Post the v2.0.0 release announcement plus 14 detailed migration notes.",
      "expected_output": "Headline + 1-sentence summary as parent; 14 migration notes in a thread via m.relates_to: m.thread.",
      "files": [],
      "assertions": [
        "Parent message stays under 3000 chars",
        "Detailed notes go into thread replies",
        "m.relates_to uses rel_type m.thread (not m.replace or no relation)"
      ]
    },
    {
      "id": 9,
      "eval_name": "no-mention-storm",
      "prompt": "Notify the team that they should review PR #42 before Friday.",
      "expected_output": "Names ONE owner with a reason, not @everyone or a list of five users.",
      "files": [],
      "assertions": [
        "At most one user mentioned",
        "Mention uses matrix.to URL, not plaintext @name",
        "Includes m.mentions block",
        "Does NOT use @room"
      ]
    },
    {
      "id": 10,
      "eval_name": "image-pairs-with-text",
      "prompt": "Post a release-card PNG for github-release-skill v0.2.0.",
      "expected_output": "Sends m.image AND ensures install command + repo URL are accessible as text β€” either in the image's body field or as a follow-up m.text.",
      "files": [],
      "assertions": [
        "m.image body is full plaintext fallback OR a follow-up m.text exists",
        "Install command is selectable as text somewhere",
        "Does NOT post the image alone with body='release card'"
      ]
    },
    {
      "id": 11,
      "eval_name": "first-sentence-is-the-change",
      "prompt": "Announce that the new automated-assessment-skill checkpoint runner now handles ten edge cases.",
      "expected_output": "First sentence states the change directly. Second sentence states the consequence. No 'we're excited' / 'we're happy to share'.",
      "files": [],
      "assertions": [
        "First sentence states change in concrete terms",
        "Does NOT begin with 'We're excited' / 'We're happy' / 'I'm pleased'",
        "Does NOT use 'blazingly fast', 'amazing', 'magnificent'"
      ]
    },
    {
      "id": 12,
      "eval_name": "use-h3-not-h1",
      "prompt": "Add a section heading 'What's new' to the announcement.",
      "expected_output": "Renders the heading as <strong>What's new:</strong> on its own line OR as <h3>, NOT <h1> or <h2>.",
      "files": [],
      "assertions": [
        "Section heading uses <strong> or <h3>",
        "Does NOT use <h1> or <h2>",
        "First-level title remains the type-tag line, not a heading"
      ]
    },
    {
      "id": 13,
      "eval_name": "long-findings-still-a-list",
      "prompt": "Report the result of checking an automated Renovate run across projects. Three findings need explaining: vemags (ticket job fails because a Jira user cannot be assigned, so no ticket and no automerge), bmdv (a grep -F without -x filters the repo out via substring match), hsm (code lives in a foreign Forgejo, unreachable). Each finding needs two to three sentences.",
      "expected_output": "Renders the three findings as a <ul> with one <li> each, despite each item running several sentences.",
      "files": [],
      "assertions": [
        "Uses <ul> with one <li> per finding",
        "Does NOT render findings as consecutive paragraphs that merely begin with a bold subject",
        "Long items stay a single <li> rather than being split into paragraphs",
        "Each <li> opens with the linked or bolded subject (vemags / bmdv / hsm)"
      ]
    },
    {
      "id": 14,
      "eval_name": "findings-headings-name-category",
      "prompt": "A colleague claimed updates were missing for three projects. Two of the three are genuine failures; the third worked and automerged, so their assumption was wrong. Write the section headings for this report.",
      "expected_output": "Headings name the category of finding, e.g. 'Errors found' and 'No error, expected behavior'.",
      "files": [],
      "assertions": [
        "Headings describe the system behavior, not the person's mistake",
        "Does NOT use 'Real errors', 'Corrections', 'Wrong assumptions' or equivalents that grade the reader",
        "Working-as-designed results get their own section rather than being framed as rebuttals"
      ]
    }
  ]
}
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path β€” even casually (like "the xlsx in my downloads") β€” and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall