simiancraft/simiancraft-skills / skills/appraise-github-issues
appraise-github-issues
VerifiedAppraise a GitHub issue backlog unattended: for each issue, decide whether it is still real and how big it is, then apply the answer to the tracker as a size label, a needs-decision or needs-human label with the question stated, or a close carrying a re-checkable receipt. A close is confirmed by a second engine before it is made. Runs once, on one issue, over the whole backlog, or as a periodic heartbeat, and is the sizing stage of burn-down-github-issues. Use when the task is "size the backlog", "appraise every issue in this repo", "close what is already done", "triage recent issues without fixing them", or "keep the backlog sized". Requires a per-repository config (its own, or the burndown's) and the codex and claude CLIs for the default seats. Skip for fixing anything; this skill never writes to the repository.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.1 KB |
| appraise.ts | script | 12.7 KB |
| lib/appraise.test.ts | script | 2.8 KB |
| lib/appraise.ts | script | 30.9 KB |
| lib/callbacks.ts | script | 4.1 KB |
| prompts/appraise.md | markdown | 6.2 KB |
| prompts/confirm-close.md | markdown | 3.3 KB |
| references/adopting.md | markdown | 4.0 KB |
| references/callbacks.md | markdown | 3.4 KB |
lib/appraise.ts
View on GitHub/**
* Appraisal: is an issue still real, and how big is it. One read-only agent turn per issue, no
* worktree, no install, no writes to the repository. The answer lands on the issue as labels, a
* comment, or a close, which is the only contract another skill (the burndown, a heartbeat, a
* person) needs: they read the tracker, never this process.
*
* A close is the one verdict that is a write with consequences, so it gets a second opinion: a
* confirmer on another engine re-checks the appraiser's receipt against the fetched base ref and
* either agrees, in which case the issue closes, or disagrees, in which case the issue goes to a
* human with both opinions on the thread. Sizing and hand-offs stay single-opinion; they are labels
* a person can change.
*
* Fail-closed rules this file enforces itself, so no prompt drift can lose them: an agent's
* answer is validated field by field and anything malformed reads as no answer; a close needs a
* confirmation written by this run, for this issue; the prior size is read from the issue, never
* from the agent; the issue's live state is re-read before anything is written; and every comment
* lands before the label that would hide the issue from selection.
*/
import { existsSync, mkdirSync, rmSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { enrichBlockers } from '../../fix-github-issue/lib/blockers.ts';
import { isTrunk, leaseLost, trackerIo } from '../../carve-github-issue/lib/claims.ts';
import { liveClaim, readTree, type Tree } from '../../carve-github-issue/lib/tree.ts';
import { logTail, readResult, renderPrompt, runAgent } from '../../fix-github-issue/lib/agent.ts';
import type { Context } from '../../fix-github-issue/lib/context.ts';
import { APPRAISAL_FILE, CONFIRMATION_FILE } from '../../fix-github-issue/lib/control-files.ts';
import { assertDistinctEngines, type Seat } from '../../fix-github-issue/lib/engines.ts';
import { appraisalCount, clearAppraisals, closeIssue, ensureLabel, recordAppraisal, sendToDlq } from '../../fix-github-issue/lib/labels.ts';
import type { Issue } from '../../fix-github-issue/lib/pipeline.ts';
import { clearSizeLabels, weighHold } from '../../fix-github-issue/lib/panel.ts';
import { isStopping, mutate, RunStopping, sh } from '../../fix-github-issue/lib/shell.ts';
import { runSizeCallback, type SizeCallbackResult } from './callbacks.ts';
export const APPRAISAL_VERDICTS = ['valid', 'already-fixed', 'obsolete', 'needs-decision', 'needs-human', 'failed'] as const;
export type AppraisalVerdict = (typeof APPRAISAL_VERDICTS)[number];
/** What the appraiser writes to `loop-appraisal.json`, after validation. */
export type AppraisalResult = {
issue: number;
verdict: AppraisalVerdict;
points?: number;
reason: string;
closeComment?: string;
};
/** What the confirmer writes to `loop-confirmation.json`, after validation. */
export type Confirmation = { issue: number; agree: boolean; reason: string };
/** What an appraisal did to the issue, for a caller that reports on it. */
export type AppraisalOutcome = {
verdict: AppraisalVerdict;
points?: number;
reason: string;
/** How the close was settled, when the verdict was a close. */
close?: 'confirmed' | 'disputed' | 'unconfirmed' | 'skipped';
/** True when the verdict changed nothing on the issue and the next run should try again. */
retry?: boolean;
/** True when the appraisal stopped because the issue changed under it; the tracker, not this outcome, says where the card is. */
changed?: boolean;
/** What the size callback did, when a directory was given and a slot matched. */
callback?: SizeCallbackResult;
/** True when the failure cap was reached and the issue went to the appraisal dead-letter queue. */
deadLetter?: boolean;
/** How a decision panel settled a `needs-decision`, when one sat: `proceed` sends the issue back to be appraised. */
panel?: 'proceed' | 'hold';
};
export type AppraiseKnobs = {
/** Only issues opened this recently are appraised. */
ageDays: number;
/** Issues appraised per run. */
appraiseLimit: number;
/** Appraisers at once; no worktree, so they are cheap. */
appraiserConcurrency: number;
/** Whether a close verdict needs the confirmer's agreement before the issue closes. */
confirmCloses: boolean;
/** Labels that keep an issue out of appraisal until a human removes them. */
skipLabels: string[];
/**
* Where size callbacks live, relative to the repository root. A producer writes `on-size-<N>`
* files there; the appraiser only looks them up. See lib/callbacks.ts for the ladder.
*/
callbacksDir: string;
/** Failed appraisals an issue may absorb before it goes to a person; kept on the issue as `loop/appraisals: N`. */
maxAppraiseAttempts: number;
/** How long a size callback's executable may run; 0 means no timer, for a callback that runs agents. */
sizeCallbackTimeoutMinutes: number;
seats: { appraiser: string; confirmer: string; callback: string };
};
export const APPRAISE_DEFAULTS: AppraiseKnobs = {
ageDays: 30,
appraiseLimit: 12,
appraiserConcurrency: 3,
confirmCloses: true,
maxAppraiseAttempts: 3,
sizeCallbackTimeoutMinutes: 0,
skipLabels: ['needs-decision', 'needs-human', 'loop/skip', 'loop/parked'],
callbacksDir: '<worktreeRoot>/appraisal-callbacks',
seats: {
appraiser: 'codex:gpt-5.6-sol',
confirmer: 'claude:claude-opus-5',
/** A callback prompt runs here; by default the appraiser's own seat. */
callback: 'codex:gpt-5.6-sol',
},
};
/** `callbacksDir` resolved: `<worktreeRoot>` expands to the project's worktree root, else the path is relative to the repository root. */
export function resolveCallbacksDir(ctx: Context, configured: string): string {
const expanded = configured.replace('<worktreeRoot>', ctx.project.worktreeRoot);
return resolve(ctx.repoRoot, expanded);
}
/**
* `confirmCloses` is the one boolean whose falsy misreadings all fail open, so it is validated
* strictly rather than coerced. Call after `loadProjectConfig`, which validates only its own knobs.
*/
export function assertConfirmCloses(value: unknown, fileName: string): void {
if (typeof value !== 'boolean') {
console.error(`config ${fileName} is invalid:\n - confirmCloses must be true or false, got ${JSON.stringify(value)}`);
process.exit(1);
}
}
const SIZE_LABEL = /^size:\s*(\d+)$/;
/** The largest size label the issue carries, or null when unsized. Max, because a relabel is add-then-remove. */
export function pointsFromLabels(labels: Array<{ name: string }>): number | null {
let points: number | null = null;
for (const label of labels) {
const match = SIZE_LABEL.exec(label.name);
if (match) points = Math.max(points ?? 0, Number(match[1]));
}
return points;
}
/** The size label names the issue carries, as they are spelled on the issue. */
function sizeLabels(labels: Array<{ name: string }>): string[] {
return labels.map((l) => l.name).filter((name) => SIZE_LABEL.test(name));
}
export const ISSUE_LIST_FIELDS = 'number,title,createdAt,labels,parent,subIssuesSummary,blockedBy';
export function allOpenIssues(ctx: Context): Issue[] {
return enrichBlockers(ctx, JSON.parse(sh(ctx, ['gh', 'issue', 'list', '--state', 'open', '--limit', '5000', '--json', ISSUE_LIST_FIELDS])) as Issue[]);
}
/** The labels that mark a trunk or a claimed issue without reading its thread. */
const TRUNK_LABELS = ['loop/carved', 'loop/released', 'loop/carving', 'loop/working'];
/** True when the listing alone says the issue is a trunk or claimed: a trunk label, or an open child. */
export function looksLikeTrunk(issue: Issue): boolean {
if (issue.labels.some((l) => TRUNK_LABELS.includes(l.name) || l.name.startsWith('loop/carve-gen'))) return true;
const summary = issue.subIssuesSummary;
return summary !== undefined && summary.total > summary.completed;
}
/**
* The population an appraiser run works: open, inside the window unless `allAges` (an issue with a
* parent is exempt from the window), not held by a skip label or the dead-letter queue, not a trunk
* or a claimed issue, and unsized unless `includeSized`. Newest first.
*/
export function selectForAppraisal(
all: Issue[],
knobs: Pick<AppraiseKnobs, 'ageDays' | 'skipLabels'>,
options: { allAges?: boolean; includeSized?: boolean } = {},
): Issue[] {
const cutoff = Date.now() - knobs.ageDays * 24 * 60 * 60 * 1000;
return all
.filter((issue) => options.allAges || issue.parent || Date.parse(issue.createdAt) >= cutoff)
.filter((issue) => !isHeld(issue.labels, knobs.skipLabels))
.filter((issue) => !looksLikeTrunk(issue))
.filter((issue) => options.includeSized || pointsFromLabels(issue.labels) === null)
.sort((a, b) => b.number - a.number);
}
/** True when a person's label holds the issue out of the loop's reach. */
export function isHeld(labels: Array<{ name: string }>, skipLabels: string[]): boolean {
return labels.some((l) => skipLabels.includes(l.name) || l.name === 'loop/dlq' || l.name.startsWith('loop/dlq:'));
}
// ---------------------------------------------------------------------------
// Validation: an agent's answer is data, and malformed data is no answer.
// ---------------------------------------------------------------------------
const nonEmptyString = (value: unknown): value is string => typeof value === 'string' && value.trim() !== '';
export function validateAppraisal(raw: unknown, issue: number): { ok: true; result: AppraisalResult } | { ok: false; why: string } {
if (!raw || typeof raw !== 'object') return { ok: false, why: 'not an object' };
const r = raw as Record<string, unknown>;
if (r.issue !== issue) return { ok: false, why: `names issue ${JSON.stringify(r.issue)}, not #${issue}` };
if (!APPRAISAL_VERDICTS.includes(r.verdict as AppraisalVerdict)) return { ok: false, why: `unknown verdict ${JSON.stringify(r.verdict)}` };
if (!nonEmptyString(r.reason)) return { ok: false, why: 'no reason' };
// A size only means something for work to do. A verdict that closes or hands the issue off may
// carry the scale's zero ("nothing to estimate"), which reads as no size rather than a bad answer.
if (r.verdict !== 'valid' && r.points === 0) r.points = undefined;
if (r.points !== undefined && r.points !== null && (!Number.isInteger(r.points) || (r.points as number) <= 0)) {
return { ok: false, why: `points must be a positive integer, got ${JSON.stringify(r.points)}` };
}
if (r.closeComment !== undefined && r.closeComment !== null && typeof r.closeComment !== 'string') {
return { ok: false, why: 'closeComment must be a string' };
}
return {
ok: true,
result: {
issue,
verdict: r.verdict as AppraisalVerdict,
points: typeof r.points === 'number' ? r.points : undefined,
reason: r.reason.trim(),
closeComment: nonEmptyString(r.closeComment) ? r.closeComment.trim() : undefined,
},
};
}
export function validateConfirmation(raw: unknown, issue: number): { ok: true; result: Confirmation } | { ok: false; why: string } {
if (!raw || typeof raw !== 'object') return { ok: false, why: 'not an object' };
const r = raw as Record<string, unknown>;
if (r.issue !== issue) return { ok: false, why: `names issue ${JSON.stringify(r.issue)}, not #${issue}` };
if (typeof r.agree !== 'boolean') return { ok: false, why: `agree must be true or false, got ${JSON.stringify(r.agree)}` };
if (!nonEmptyString(r.reason)) return { ok: false, why: 'no reason' };
return { ok: true, result: { issue, agree: r.agree, reason: r.reason.trim() } };
}
// ---------------------------------------------------------------------------
// The two agent turns
// ---------------------------------------------------------------------------
/**
* Asks a second engine whether the appraiser's close holds. Returns the confirmation, or null when
* the confirmer crashed, wrote nothing, or wrote something malformed; none of those is evidence.
* The scratch directory is unique to this process so two drivers cannot read each other's answer.
*/
export async function confirmClose(
ctx: Context,
issue: Issue,
appraisal: Pick<AppraisalResult, 'verdict' | 'reason' | 'closeComment'>,
confirmer: Seat,
say: (message: string) => void,
): Promise<Confirmation | null> {
const cwd = join(ctx.runDir, `confirm-${issue.number}-${process.pid}`);
rmSync(cwd, { recursive: true, force: true });
mkdirSync(cwd, { recursive: true });
const prompt = renderPrompt(ctx, 'confirm-close.md', {
ISSUE: String(issue.number),
TITLE: issue.title,
VERDICT: appraisal.verdict,
APPRAISER_REASON: appraisal.reason,
CLOSE_COMMENT: appraisal.closeComment ?? appraisal.reason,
});
const run = await runAgent(ctx, 'confirmer', issue.number, cwd, confirmer, prompt);
if (run.exitCode !== 0) {
rmSync(cwd, { recursive: true, force: true });
return null;
}
const checked = validateConfirmation(readResult<unknown>(cwd, CONFIRMATION_FILE), issue.number);
if (!checked.ok) {
say(`confirmer's answer is unusable (${checked.why}); kept at ${cwd}`);
return null;
}
rmSync(cwd, { recursive: true, force: true });
return checked.result;
}
const ANNOUNCEMENT_FRESH_MS = 30 * 60 * 1000;
/** True when this account announced the same hand-off on this open issue within the last thirty minutes. */
function recentAnnouncement(ctx: Context, issue: number, marker: string): boolean {
if (ctx.dryRun) return false;
const node = trackerIo(ctx).view(issue);
if (!node || node.state !== 'OPEN') return false;
return node.comments.some((c) => c.author === ctx.botLogin && c.body.startsWith(marker) && Date.now() - Date.parse(c.createdAt) < ANNOUNCEMENT_FRESH_MS);
}
/** The hand-off intent: the announcement with its payload, then the hold label unless `announceOnly`. */
function handOff(ctx: Context, issue: number, verdict: 'needs-decision' | 'needs-human', reason: string, extra: Record<string, unknown> = {}, announceOnly = false): void {
const marker = `<!-- appraise-handoff verdict=${verdict} -->`;
if (!recentAnnouncement(ctx, issue, marker)) {
const body = [marker, '```json', JSON.stringify({ verdict, reason, ...extra }), '```', reason].join('\n');
mutate(ctx, `comment on #${issue}`, ['gh', 'issue', 'comment', String(issue), '--body', body]);
}
if (announceOnly) return;
mutate(ctx, `label #${issue} ${verdict}`, ['gh', 'issue', 'edit', String(issue), '--add-label', verdict]);
}
/**
* A driver's appraisal lane threw (a fetch, a worktree, a tracker write). That is a failed
* appraisal like any other: counted, and at the cap a dead letter, or the issue would be asked
* again on every run with nothing accumulating against it.
*/
export function recordAppraisalThrow(ctx: Context, issue: Issue, cap: number | undefined, error: Error, say: (m: string) => void): AppraisalOutcome {
return countFailedAppraisal(ctx, issue, cap ?? APPRAISE_DEFAULTS.maxAppraiseAttempts, `the appraisal threw: ${error.message.split('\n')[0]}`, null, say);
}
/**
* A trunk is not appraised, with one exception the burndown asks for: the released trunk whose
* remainder it wants sized or closed. Every gate in an appraisal asks this, not `isTrunk`, or the
* exception admitted at the start is refused at the end.
*/
export function refusedAsTrunk(tree: Tree, release: boolean | undefined): boolean {
if (!isTrunk(tree)) return false;
return !(release && tree.issue.labels.some((l) => l.name === 'loop/released'));
}
/** One more failed appraisal; at the cap the issue goes to the appraisal dead-letter queue with the log tail. */
function countFailedAppraisal(ctx: Context, issue: Issue, cap: number, reason: string, logPath: string | null, say: (m: string) => void): AppraisalOutcome {
if (ctx.dryRun) return { verdict: 'failed', reason, retry: true };
// A stop is the operator's, not the appraiser's: the turn is not a failed one, and nothing about
// it is settled, the card included, so it unwinds the same way the agent's own stop does.
if (isStopping()) throw new RunStopping(`the run is stopping; not counted: ${reason}`);
// A release appraisal runs under a lease. Once that is lost the count is another run's to keep.
if (leaseLost(ctx, issue.number)) {
say(`not counted, since this run lost its lease on the issue: ${reason}`);
return { verdict: 'failed', reason, retry: true, changed: true };
}
const attempts = recordAppraisal(ctx, issue.number, appraisalCount(issue.labels));
say(`appraisal attempt ${attempts} of ${cap} failed: ${reason}`);
if (attempts < cap) return { verdict: 'failed', reason, retry: true };
const tail = logPath ? logTail(logPath) : 'no log';
// A machine gave up, which is a dead letter and not a person's question: nothing about the
// issue needs deciding, the appraiser needs another seat, a changed thread, or a cooling period.
sendToDlq(ctx, issue.number, 'appraisal', `The appraiser or its confirmer failed ${attempts} times on this issue and stops trying. Last failure: ${reason}. Log tail: ${tail}`);
return { verdict: 'failed', reason: `appraisal dead letter after ${attempts} failures: ${reason}`, retry: false, deadLetter: true };
}
/**
* Appraises one issue and applies the verdict to the tracker. `onVerdict` fires once the
* appraiser has answered and before anything is written, for a caller that keeps a board.
*/
export async function appraiseIssue(
ctx: Context,
issue: Issue,
options: {
/** The window the driver applied; null when it applied none. Rendered into the prompt. */
ageDays: number | null;
seats: { appraiser: Seat; confirmer: Seat };
confirmCloses: boolean;
skipLabels: string[];
maxAppraiseAttempts?: number;
sizeCallbackTimeoutMinutes?: number;
/** Where a producer put its size callbacks, and the seat a callback prompt runs on. */
callbacks?: { dir: string; seat: Seat };
/** Accept the one trunk carrying `loop/released`, size its remainder, and fire no size callback. */
release?: boolean;
/** The runId whose claim is not foreign; the burndown's release appraisal runs under its own. */
ownClaim?: string;
onVerdict?: (outcome: AppraisalOutcome) => void;
},
): Promise<AppraisalOutcome> {
const say = (message: string) => ctx.log(`#${issue.number} ${message}`);
const cap = options.maxAppraiseAttempts ?? APPRAISE_DEFAULTS.maxAppraiseAttempts;
// The confirmer is the second opinion on a close; with closes on the appraiser alone it never runs.
if (options.confirmCloses) assertDistinctEngines(options.seats.appraiser, options.seats.confirmer, 'appraiser and confirmer');
// A trunk is worked by closing its children, never re-sized while they are open; the one
// exception is a released trunk the burndown asks this to finish.
const io = trackerIo(ctx);
const before = readTree(ctx, issue.number, io);
if (refusedAsTrunk(before, options.release)) {
say('a trunk is not appraised; it is worked by closing its children');
return { verdict: 'failed', reason: 'a trunk is not appraised', retry: false };
}
const cwd = join(ctx.runDir, `appraise-${issue.number}-${process.pid}`);
rmSync(cwd, { recursive: true, force: true });
mkdirSync(cwd, { recursive: true });
const prompt = renderPrompt(ctx, 'appraise.md', {
ISSUE: String(issue.number),
TITLE: issue.title,
AGE_DAYS: options.ageDays === null ? 'any' : String(options.ageDays),
});
const run = await runAgent(ctx, 'appraiser', issue.number, cwd, options.seats.appraiser, prompt);
if (run.exitCode !== 0) {
say(`appraiser exited ${run.exitCode}; its verdict is not trusted`);
rmSync(cwd, { recursive: true, force: true });
return countFailedAppraisal(ctx, issue, cap, `appraiser exited ${run.exitCode}`, run.logPath, say);
}
if (ctx.dryRun) {
rmSync(cwd, { recursive: true, force: true });
return { verdict: 'failed', reason: 'dry run; no appraiser ran', retry: true };
}
const checked = validateAppraisal(readResult<unknown>(cwd, APPRAISAL_FILE), issue.number);
if (!checked.ok) {
say(`appraiser's answer is unusable (${checked.why}); scratch kept at ${cwd}`);
return countFailedAppraisal(ctx, issue, cap, `no usable appraisal: ${checked.why}`, run.logPath, say);
}
const result = checked.result;
if (result.verdict === 'valid' && result.points !== undefined && !ctx.knobs.pointScale.includes(result.points)) {
say(`appraiser sized it ${result.points}, which is not on the scale ${ctx.knobs.pointScale.join(', ')}`);
return countFailedAppraisal(ctx, issue, cap, `size ${result.points} is not on the scale`, run.logPath, say);
}
// A valid issue with no size is half a verdict, and half a verdict is a failed turn: counted,
// or an appraiser that never sizes this issue would be asked again on every run, for ever.
if (result.verdict === 'valid' && !result.points) {
say('appraiser called it valid but gave no size');
return countFailedAppraisal(ctx, issue, cap, 'valid, with no size', run.logPath, say);
}
say(`appraisal: ${result.verdict}${result.points ? ` at ${result.points} points` : ''}; ${result.reason}`);
const outcome: AppraisalOutcome = { verdict: result.verdict, points: result.points, reason: result.reason };
options.onVerdict?.(outcome);
// The selection was a snapshot; a person, another driver, or a merge may have moved the issue
// while the appraiser read it. Apply the verdict to the issue as it is now, or not at all.
const now = readTree(ctx, issue.number, io);
const live = { state: now.issue.state, labels: now.issue.labels };
if (live.state !== 'OPEN') {
say(`closed while being appraised; nothing applied`);
rmSync(cwd, { recursive: true, force: true });
return { ...outcome, verdict: 'failed', reason: 'issue closed while being appraised', changed: true };
}
if (isHeld(live.labels, options.skipLabels)) {
say(`a hold label landed while it was being appraised; nothing applied`);
rmSync(cwd, { recursive: true, force: true });
return { ...outcome, verdict: 'failed', reason: 'issue held while being appraised', changed: true };
}
if (refusedAsTrunk(now, options.release)) {
say('became a trunk while being appraised; nothing applied');
rmSync(cwd, { recursive: true, force: true });
return { ...outcome, verdict: 'failed', reason: 'issue became a trunk while being appraised', retry: false, changed: true };
}
const foreign = liveClaim(now, new Date().toISOString(), options.ownClaim ?? ctx.runId);
if (foreign) {
say(`claimed by ${foreign.runId} while being appraised; nothing applied`);
rmSync(cwd, { recursive: true, force: true });
return { ...outcome, verdict: 'failed', reason: `issue claimed by ${foreign.runId} while being appraised`, retry: true, changed: true };
}
// A lost lease may be another run's by now, whether or not its claim is visible yet.
if (leaseLost(ctx, issue.number)) {
say('this run lost its lease while it was being appraised; nothing applied');
rmSync(cwd, { recursive: true, force: true });
return { ...outcome, verdict: 'failed', reason: 'this run lost its lease on the issue while it was being appraised', retry: true, changed: true };
}
const priorSizes = sizeLabels(live.labels);
const priorPoints = pointsFromLabels(live.labels);
// A verdict that lands clears the failure count; the next failure starts a fresh budget.
if (result.verdict !== 'failed' && appraisalCount(live.labels) > 0) clearAppraisals(ctx, issue.number);
switch (result.verdict) {
case 'already-fixed':
case 'obsolete': {
const closeComment = result.closeComment ?? result.reason;
if (!options.confirmCloses) {
await closeIssue(ctx, issue.number, closeComment, { kind: 'closed', reason: result.verdict, by: 'appraiser' });
outcome.close = 'skipped';
break;
}
// The appraiser is the seat here, so it moves the card; a board write never fails an appraisal.
try {
ctx.onLane?.({ issue: issue.number, title: issue.title, lane: 'A3', note: `a second engine re-checks the ${result.verdict}` });
} catch (error) {
say(`board: ${(error as Error).message}`);
}
const confirmation = await confirmClose(ctx, issue, result, options.seats.confirmer, say);
if (!confirmation) {
say('no usable confirmation; the close is not made and the issue stays unsized');
// Counted like any failed appraisal, or a confirmer that always fails would retry forever.
const counted = countFailedAppraisal(ctx, issue, cap, 'the close confirmer produced no usable answer', null, say);
outcome.close = 'unconfirmed';
outcome.retry = counted.retry;
outcome.deadLetter = counted.deadLetter;
if (counted.deadLetter) outcome.reason = counted.reason;
break;
}
// The confirmer's turn took minutes, and the lease can have run out under it.
if (leaseLost(ctx, issue.number)) {
say('this run lost its lease while the close was being confirmed; nothing applied');
outcome.close = 'unconfirmed';
outcome.retry = true;
outcome.changed = true;
break;
}
if (confirmation.agree) {
// The confirmer's turn took minutes. A hold, a claim, a child, or a close can have landed
// on the issue meanwhile, and a close is the one write here nobody can take back quietly.
const late = readTree(ctx, issue.number, io);
const lateClaim = liveClaim(late, new Date().toISOString(), options.ownClaim ?? ctx.runId);
const lateWhy =
late.issue.state !== 'OPEN'
? 'it was closed'
: isHeld(late.issue.labels, options.skipLabels)
? 'a hold label landed'
: refusedAsTrunk(late, options.release)
? 'it became a trunk'
: lateClaim
? `${lateClaim.runId} claimed it`
: null;
if (lateWhy) {
say(`${lateWhy} while the close was being confirmed; nothing applied`);
outcome.close = 'unconfirmed';
outcome.retry = true;
outcome.changed = true;
break;
}
say(`confirmer agrees: ${confirmation.reason}`);
await closeIssue(ctx, issue.number, `${closeComment}\n\nIndependently re-checked: ${confirmation.reason}`, { kind: 'closed', reason: result.verdict, by: 'appraiser' });
outcome.close = 'confirmed';
} else {
say(`confirmer disagrees: ${confirmation.reason}`);
// Comment first, then the label that hides the issue from selection, so a crash between
// the two leaves an issue that is still selectable rather than one held without a reason.
mutate(ctx, `comment on #${issue.number}`, [
'gh',
'issue',
'comment',
String(issue.number),
'--body',
[
`Appraisal proposed closing this as \`${result.verdict}\`, and an independent re-check disagreed; a person needs to settle it.`,
'',
`**Proposed close:** ${closeComment}`,
'',
`**Re-check:** ${confirmation.reason}`,
].join('\n'),
]);
mutate(ctx, `label #${issue.number} needs-human`, ['gh', 'issue', 'edit', String(issue.number), '--add-label', 'needs-human']);
outcome.close = 'disputed';
}
break;
}
case 'needs-decision': {
// The question goes on the thread first; a decision panel, when one can sit, weighs it there
// before the hold is applied, and an issue it settles goes back to be appraised instead.
handOff(ctx, issue.number, result.verdict, result.reason, {}, true);
const weighed = await weighHold(ctx, { issue: issue.number, title: issue.title, question: result.reason, raiser: options.seats.appraiser });
if (weighed.kind === 'unavailable') say(`no decision panel (${weighed.why}); held for a person`);
else outcome.panel = weighed.kind;
if (weighed.kind === 'proceed') {
clearSizeLabels(ctx, issue.number);
break;
}
// The panel's turns took minutes; a person may have closed or held the issue meanwhile.
const late = readTree(ctx, issue.number, io);
if (late.issue.state !== 'OPEN' || isHeld(late.issue.labels, options.skipLabels)) {
say('closed or held while the question was weighed; no hold applied');
outcome.changed = true;
break;
}
mutate(ctx, `label #${issue.number} needs-decision`, ['gh', 'issue', 'edit', String(issue.number), '--add-label', 'needs-decision']);
break;
}
case 'needs-human':
handOff(ctx, issue.number, result.verdict, result.reason);
break;
case 'valid':
if (!result.points) break; // refused above; this narrows the type
if (priorPoints === result.points) {
say(`already sized at ${result.points}; nothing to change`);
} else {
// Add the new size and only then note the disagreement; the prior label is left in place
// because a person put it there, and `pointsFromLabels` reads the larger of the two.
// A repository that has never been appraised has no size labels, and a label the
// repository lacks cannot be put on an issue.
ensureLabel(ctx, `size: ${result.points}`, 'c5def5', `Sized at ${result.points} by an appraisal`);
mutate(ctx, `size #${issue.number} at ${result.points}`, ['gh', 'issue', 'edit', String(issue.number), '--add-label', `size: ${result.points}`]);
if (priorSizes.length > 0) {
mutate(ctx, `note the sizing disagreement on #${issue.number}`, [
'gh',
'issue',
'comment',
String(issue.number),
'--body',
`Re-sized at ${result.points} points, previously \`${priorSizes.join('`, `')}\`. ${result.reason}`,
]);
}
}
// The size is on the issue; what happens next for an issue of this size is the producer's.
// A released trunk fires no callback: the burndown that asked for the release appraisal
// carves an oversized remainder itself, under the claim it already holds.
if (options.callbacks && !options.release) {
// A callback can be another driver, and a long one: say what is running before it starts,
// so the log does not go silent while a carve happens somewhere else.
say(`handing #${issue.number} to the ${result.points}-point size callback in ${options.callbacks.dir}; its own log is where that callback writes one`);
outcome.callback = await runSizeCallback(
ctx,
options.callbacks.dir,
options.callbacks.seat,
issue,
{
issue: issue.number,
title: issue.title,
points: result.points,
priorPoints,
verdict: 'valid',
reason: result.reason,
repo: ctx.project.repo,
baseBranch: ctx.project.baseBranch,
repoRoot: ctx.repoRoot,
},
say,
{ timeoutMs: (options.sizeCallbackTimeoutMinutes ?? APPRAISE_DEFAULTS.sizeCallbackTimeoutMinutes) * 60_000 },
);
}
break;
case 'failed':
if (existsSync(cwd)) rmSync(cwd, { recursive: true, force: true });
return countFailedAppraisal(ctx, issue, cap, result.reason, run.logPath, say);
}
if (existsSync(cwd)) rmSync(cwd, { recursive: true, force: true });
return outcome;
}
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |