Skip to content

simiancraft/simiancraft-skills / skills/appraise-github-issues

appraise-github-issues

Verified

Appraise a GitHub issue backlog unattended: for each issue, decide whether it is still real and how big it is, then apply the answer to the tracker as a size label, a needs-decision or needs-human label with the question stated, or a close carrying a re-checkable receipt. A close is confirmed by a second engine before it is made. Runs once, on one issue, over the whole backlog, or as a periodic heartbeat, and is the sizing stage of burn-down-github-issues. Use when the task is "size the backlog", "appraise every issue in this repo", "close what is already done", "triage recent issues without fixing them", or "keep the backlog sized". Requires a per-repository config (its own, or the burndown's) and the codex and claude CLIs for the default seats. Skip for fixing anything; this skill never writes to the repository.

FileKindSize
SKILL.mdskill md7.1 KB
appraise.tsscript12.7 KB
lib/appraise.test.tsscript2.8 KB
lib/appraise.tsscript30.9 KB
lib/callbacks.tsscript4.1 KB
prompts/appraise.mdmarkdown6.2 KB
prompts/confirm-close.mdmarkdown3.3 KB
references/adopting.mdmarkdown4.0 KB
references/callbacks.mdmarkdown3.4 KB

appraise.ts

View on GitHub
/**
 * appraise-github-issues: size the backlog, close what is done, hand off what needs a person.
 *
 * One read-only agent turn per issue answers two questions, is it still real and how big is it,
 * and the answer lands on the issue as a `size: N` label, a `needs-decision` or `needs-human`
 * label with the question stated, or a close with a re-checkable receipt. A close is confirmed by
 * a second engine before it is made. This file is the command; the library it drives is in `lib/`,
 * and another driver (the issue burndown) calls `appraiseIssue` directly.
 *
 * This file is shared: it ships with the skill and is not copied into a repository. Everything
 * true of a repository lives in a config file at that repository's root. Run it from inside the
 * target repository:
 *
 *   bun run <skill-dir>/appraise.ts --dry-run             # select and print; no agent, no mutation
 *   bun run <skill-dir>/appraise.ts --limit 12            # appraise up to 12 unsized issues in the window
 *   bun run <skill-dir>/appraise.ts --issue <n>           # one issue, whatever its age or size
 *   bun run <skill-dir>/appraise.ts --all                 # the whole open backlog, not only the window
 *   bun run <skill-dir>/appraise.ts --include-sized       # re-judge issues that already carry a size
 *   bun run <skill-dir>/appraise.ts --every <minutes>     # a heartbeat: appraise whatever is new, on a cadence you choose
 *   bun run <skill-dir>/appraise.ts --no-confirm          # close on the appraiser's word alone
 *   bun run <skill-dir>/appraise.ts --appraiser codex:gpt-5.6-sol --confirmer claude:claude-opus-5
 *   bun run <skill-dir>/appraise.ts --callbacks <dir>     # where a producer's on-size-<N> callbacks live
 */

import { existsSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { enrichBlockers } from '../fix-github-issue/lib/blockers.ts';
import { invokeRootFrom, loadProjectConfig, PIPELINE_DEFAULTS, type PipelineKnobs, repoRootFrom } from '../fix-github-issue/lib/config.ts';
import { createContext } from '../fix-github-issue/lib/context.ts';
import { parseSeat, seatLabel } from '../fix-github-issue/lib/engines.ts';
import { claimLock } from '../fix-github-issue/lib/lane.ts';
import { ensureLabels } from '../fix-github-issue/lib/labels.ts';
import type { Issue } from '../fix-github-issue/lib/pipeline.ts';
import { pool } from '../fix-github-issue/lib/pool.ts';
import { log, sh, step, teeConsole } from '../fix-github-issue/lib/shell.ts';
import {
  allOpenIssues,
  APPRAISE_DEFAULTS,
  type AppraiseKnobs,
  appraiseIssue,
  assertConfirmCloses,
  isHeld,
  resolveCallbacksDir,
  selectForAppraisal,
  ISSUE_LIST_FIELDS,
} from './lib/appraise.ts';
import { existsSync as dirExists } from 'node:fs';
import { installStopHandler } from '../fix-github-issue/lib/stop.ts';
import { guardCli } from '../fix-github-issue/lib/cli.ts';

const HERE = dirname(fileURLToPath(import.meta.url));
const PROMPTS = join(HERE, 'prompts');

const args = process.argv.slice(2);

/** Every flag this command reads is named here and in USAGE; a source test holds the three together. */
export const CLI = { flags: ['dry-run', 'all', 'include-sized', 'no-confirm'], options: ['issue', 'limit', 'age-days', 'every', 'appraiser', 'confirmer', 'callbacks', 'callback-seat'] } as const;
export const USAGE = `appraise-github-issues: size the open issues nobody has sized, and hand off the ones that are not work.
Run it from inside the target repository. With no flags it starts a REAL pass over the window.

  bun run <skill-dir>/appraise.ts [flags]

  --dry-run               select and print; no agent, no mutation
  --issue <n>             one issue, whatever its age or size
  --limit <n>             appraise at most n unsized issues in the window (default: the config's)
  --age-days <n>          the window, in days (default: the config's)
  --all                   the whole open backlog, not only the window
  --include-sized         re-judge issues that already carry a size
  --every <minutes>       a heartbeat: appraise whatever is new, on that cadence
  --no-confirm            close on the appraiser's word alone
  --appraiser, --confirmer, --callback-seat <seat>
                          engine[:model] for that seat
  --callbacks <dir>       where a producer's on-size-<N> callbacks live
  --help, -h              print this and exit`;
// Before anything else: an argument this command does not know must never fall through to a real run.
guardCli(args, CLI, USAGE);
const flag = (name: string) => args.includes(`--${name}`);
/** The value after `--name`; a present flag with no value (end of argv, or another flag) is an error, not an absence. */
const opt = (name: string) => {
  const i = args.indexOf(`--${name}`);
  if (i < 0) return undefined;
  const value = args[i + 1];
  if (value === undefined || value.startsWith('--')) {
    console.error(`--${name} needs a value`);
    process.exit(1);
  }
  return value;
};
const positive = (name: string, fallback: number): number => {
  const raw = opt(name);
  if (raw === undefined) return fallback;
  const parsed = Number(raw);
  if (!Number.isInteger(parsed) || parsed <= 0) {
    console.error(`--${name} expects a positive integer, got '${raw}'`);
    process.exit(1);
  }
  return parsed;
};

const DRY_RUN = flag('dry-run');
const ALL_AGES = flag('all');
const INCLUDE_SIZED = flag('include-sized');
const NO_CONFIRM = flag('no-confirm');
const ISSUE_NUMBER = opt('issue') !== undefined ? positive('issue', 0) : null;
const EVERY = opt('every') !== undefined ? positive('every', 0) : null;

const REPO_ROOT = repoRootFrom(process.cwd());
const INVOKE_ROOT = invokeRootFrom(process.cwd(), REPO_ROOT);

/**
 * Its own config when the repository has one, the burndown's otherwise, reading only the fields it
 * needs. An adopter of the burndown gets this command without writing a second file.
 */
const CONFIG_FILE = ['appraise-github-issues.config.ts', 'burn-down-github-issues.config.ts'].find((name) =>
  existsSync(join(INVOKE_ROOT, name)),
);

const CONFIG = await loadProjectConfig<PipelineKnobs & AppraiseKnobs & { seats: AppraiseKnobs['seats'] & PipelineKnobs['seats'] }>({
  invokeRoot: INVOKE_ROOT,
  repoRoot: REPO_ROOT,
  fileName: CONFIG_FILE ?? 'appraise-github-issues.config.ts',
  // The loader validates the pipeline's own knobs whatever the caller, so they ride along with defaults.
  defaults: { ...PIPELINE_DEFAULTS, ...APPRAISE_DEFAULTS, seats: { ...PIPELINE_DEFAULTS.seats, ...APPRAISE_DEFAULTS.seats } },
  positiveIntegers: ['ageDays', 'appraiseLimit', 'appraiserConcurrency', 'maxReviewRounds', 'maxAppraiseAttempts'],
  nonNegativeIntegers: ['sizeCallbackTimeoutMinutes'],
  help: [
    'Appraisal is shared across repositories; everything true of a repository lives in that file.',
    'Copy the template from references/adopting.md in this skill (or adopt burn-down-github-issues, whose config this command also reads).',
  ],
});

assertConfirmCloses(CONFIG.confirmCloses, CONFIG_FILE ?? 'appraise-github-issues.config.ts');
const LIMIT = positive('limit', CONFIG.appraiseLimit);
const AGE_DAYS = positive('age-days', CONFIG.ageDays);

const SEATS = (() => {
  try {
    return {
      appraiser: parseSeat(opt('appraiser') ?? CONFIG.seats.appraiser, '--appraiser'),
      confirmer: parseSeat(opt('confirmer') ?? CONFIG.seats.confirmer, '--confirmer'),
      callback: parseSeat(opt('callback-seat') ?? CONFIG.seats.callback, '--callback-seat'),
    };
  } catch (error) {
    console.error((error as Error).message);
    process.exit(1);
  }
})();

const ctx = createContext({
  project: CONFIG.project,
  knobs: { autoMerge: 'never', maxReviewRounds: 1, checksTimeoutMinutes: CONFIG.checksTimeoutMinutes, agentTimeoutMinutes: CONFIG.agentTimeoutMinutes, smokeTimeoutMinutes: CONFIG.smokeTimeoutMinutes, pointScale: CONFIG.pointScale },
  seats: {
    worker: parseSeat(CONFIG.seats.worker, 'seats.worker'),
    reviewer: parseSeat(CONFIG.seats.reviewer, 'seats.reviewer'),
  },
  repoRoot: REPO_ROOT,
  invokeRoot: INVOKE_ROOT,
  promptsDirs: [PROMPTS],
  dryRun: DRY_RUN,
});

const CALLBACKS_DIR = opt('callbacks') ? resolve(opt('callbacks') as string) : resolveCallbacksDir(ctx, CONFIG.callbacksDir);

teeConsole(join(ctx.runDir, 'appraise.log'));
step(`${ctx.project.name} appraise-github-issues`);
log(`run pid ${process.pid} started ${new Date().toISOString()}`);
log(`base ${ctx.project.baseBranch} | ${ALL_AGES ? 'all ages' : `last ${AGE_DAYS} days`} | ${INCLUDE_SIZED ? 'sized and unsized' : 'unsized only'} | config ${CONFIG_FILE}`);
log(`appraiser ${seatLabel(SEATS.appraiser)} | confirmer ${seatLabel(SEATS.confirmer)} | closes ${NO_CONFIRM || CONFIG.confirmCloses === false ? 'on the appraiser alone' : 'confirmed by the second engine'}`);
log(`size callbacks: ${CALLBACKS_DIR}${dirExists(CALLBACKS_DIR) ? '' : ' (absent; none will run)'} | callback seat ${seatLabel(SEATS.callback)}`);
// Refused before any issue is read: every appraisal would otherwise throw on its own.
if (!(NO_CONFIRM || CONFIG.confirmCloses === false) && SEATS.appraiser.engine === SEATS.confirmer.engine) {
  console.error(`appraiser and confirmer must be different engines; both are ${SEATS.appraiser.engine}. Pick another --confirmer, or pass --no-confirm.`);
  process.exit(2);
}
if (DRY_RUN) log('DRY RUN: selection only; no lock, no agent, no GitHub mutation; only this log is written');

/** A dry run stops at selection: it prints what a real run would appraise and touches nothing else. */
if (DRY_RUN) {
  const selected = ISSUE_NUMBER
    ? [JSON.parse(sh(ctx, ['gh', 'issue', 'view', String(ISSUE_NUMBER), '--json', ISSUE_LIST_FIELDS])) as Issue]
    : selectForAppraisal(allOpenIssues(ctx), { ageDays: AGE_DAYS, skipLabels: CONFIG.skipLabels }, { allAges: ALL_AGES, includeSized: INCLUDE_SIZED }).slice(0, LIMIT);
  const held = selected.filter((issue) => isHeld(issue.labels, CONFIG.skipLabels));
  for (const issue of held) log(`#${issue.number} carries a hold label; a person holds it, so it would not be appraised`);
  const would = selected.filter((issue) => !isHeld(issue.labels, CONFIG.skipLabels));
  log(would.length === 0 ? 'would appraise nothing' : `would appraise ${would.map((i) => `#${i.number}`).join(', ')}`);
  step('done');
  process.exit(0);
}

const releaseLock = (() => {
  try {
    return claimLock(ctx, 'appraise.lock');
  } catch (error) {
    log((error as Error).message);
    process.exit(1);
  }
})();
process.on('exit', releaseLock);
installStopHandler(log, releaseLock);

ensureLabels(ctx);

async function once(): Promise<void> {
  // Appraisers judge "already in the base" against the fetched base ref, not the main checkout's
  // working state, which may be stale, dirty, or on another branch.
  sh(ctx, ['git', 'fetch', ctx.project.remote, ctx.project.baseBranch]);

  let issues: Issue[];
  if (ISSUE_NUMBER) {
    // Bypasses the window and the size filter, never the labels a person set to hold an issue.
    const one: Issue = JSON.parse(sh(ctx, ['gh', 'issue', 'view', String(ISSUE_NUMBER), '--json', ISSUE_LIST_FIELDS]));
    enrichBlockers(ctx, [one]);
    if (isHeld(one.labels, CONFIG.skipLabels)) {
      log(`#${one.number} carries a hold label; a person holds it, so it is not appraised. Remove the label to put it back in reach.`);
      return;
    }
    issues = [one];
  } else {
    issues = selectForAppraisal(allOpenIssues(ctx), { ageDays: AGE_DAYS, skipLabels: CONFIG.skipLabels }, {
      allAges: ALL_AGES,
      includeSized: INCLUDE_SIZED,
    }).slice(0, LIMIT);
  }
  if (issues.length === 0) {
    log('nothing to appraise; the population is fully sized');
    return;
  }
  step(`Appraising ${issues.length} issue(s), ${CONFIG.appraiserConcurrency} at a time`);
  log(issues.map((i) => `#${i.number}`).join(', '));
  const tally = new Map<string, number>();
  await pool(
    issues,
    CONFIG.appraiserConcurrency,
    async (issue) => {
      const outcome = await appraiseIssue(ctx, issue, {
        ageDays: ALL_AGES ? null : AGE_DAYS,
        seats: SEATS,
        confirmCloses: !(NO_CONFIRM || CONFIG.confirmCloses === false),
        skipLabels: CONFIG.skipLabels,
        maxAppraiseAttempts: CONFIG.maxAppraiseAttempts,
        sizeCallbackTimeoutMinutes: CONFIG.sizeCallbackTimeoutMinutes,
        callbacks: { dir: CALLBACKS_DIR, seat: SEATS.callback },
      });
      const key = outcome.close
        ? `${outcome.verdict} (${outcome.close})`
        : outcome.callback?.name
          ? `${outcome.verdict} (${outcome.callback.name})`
          : outcome.verdict;
      tally.set(key, (tally.get(key) ?? 0) + 1);
    },
    (issue) => `#${issue.number}`,
  );
  log([...tally.entries()].map(([k, n]) => `${n} ${k}`).join(', '));
}

if (EVERY === null) {
  await once();
  step('done');
} else {
  for (;;) {
    await once();
    log(`sleeping ${EVERY} minute(s)`);
    await Bun.sleep(EVERY * 60 * 1000);
  }
}
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall