akiiwa-games/AgentCompass / skills/agent-compass
agent-compass
FlaggedChoose owner, model and effort, outbound authorization, advisor use, and continuation for delegated work. Reads local usage and effective account policy, profiles the task with the configured System-One scorer, then allocates a compatible worker. Use for independent worker or advisor work; ordinary local execution needs no routing.
scripts/candidate_eval.py
View on GitHub"""Candidate evaluation: one bundle against one task, its gates, exclusions (rule B) and breakdown."""
import cwd_policy
import fast_modes
import host_herdr
import host_policy
import launch_failures
import live_models
import model_latch
import provider_retry
import user_scores
import accounts_override
import auto_bundles
import account_fleet
import axis_scores
import quota_economy
import quota_overrides
import quota_pools
import task_intake
import worker_sessions
from model_resolution import model_pin_problem, split_model_effort
METERED_LAST_RESORT = 'metered access is a last resort'
QUOTA_READING_UNAVAILABLE = 'quota reading unavailable (no current observation of any window of this account, so its spend cannot be priced)'
# Round 15 (owner): every quota account has a weekly (long reset-period) window; one without is a surfaced data error, never a
# 5-hour-only account that stands in for other accounts' work as a free outside point.
QUOTA_LONG_UNREAD = ('weekly quota reading unavailable (the account has a long reset-period window but no current or last-good '
'observation of it, only its 5-hour window is read, so its spend cannot be priced)')
QUOTA_NO_LONG_WINDOW = ('no weekly quota window (neither the account configuration nor the public capacity data gives this '
'account a long reset-period window, so its spend cannot be priced)')
QUOTA_DATA_ERRORS = (QUOTA_READING_UNAVAILABLE, QUOTA_LONG_UNREAD, QUOTA_NO_LONG_WINDOW)
# The two exclusions that are the user's routing preferences rather than facts. A route the user names lifts
# OUT_OF_AUTOMATIC_ROUTING (host policy governs only what the router chooses by itself); an explicit resume of an
# existing worker lifts both, since continuing a worker is not choosing its account or model (dispatch_registry.prepare).
OUT_OF_AUTOMATIC_ROUTING = 'excluded by host policy'
UNCHECKED = 'unchecked in model management'
# The user's local settings that exclude a candidate, by kind (`local_settings`): host policy's allowed_accounts, the
# model-management checkbox, and the account's cwd_regex. Only the first two are lifted by a route the user names
# (route_judge.record_constraint refuses a named route outside cwd_regex).
LIFTED_BY_NAMING = ('allowed_accounts', 'checkbox')
# A declared dispatch cadence (quota_pools.cadence_window_state) is a pace of automatic routing, counted from this
# machine's launches, not a fact of the account: at its limit it excludes a candidate (OVER_CADENCE) or, when a seat frees
# soon, defers it on the window CADENCE.
OVER_CADENCE = 'over the dispatch cadence of automatic routing'
CADENCE = 'cadence'
# What only automatic routing is held to. A route the user names, and an explicit resume, reach past each:
# dispatch_registry.prepare lifts these exclusions by value and does not wait for these windows (router-internals.md,
# Pins). The account and model switches are lifted earlier too, for a named bundle (evaluate_bundle `user_named`).
AUTOMATIC_ROUTING_EXCLUSIONS = (OUT_OF_AUTOMATIC_ROUTING, UNCHECKED, OVER_CADENCE)
AUTOMATIC_ROUTING_WAITS = (CADENCE,)
def said_exclusions(candidate):
"""What a caller reads for each exclusion of `candidate`: the two routing preferences in their owner's words
(`host_policy.outside_routing`, `unchecked_routing`), naming the account or model and the setting, and that it is
the user's own setting on this machine; every other exclusion as written. `excluded` keeps the two constants:
`dispatch_registry.prepare` lifts them by value. A row `disabled` other than by the user's checkbox (`unchecked_by`)
says its own fact: off by default in the shipped data (`auto_bundles.off_by_default`), or not seen by discovery,
which its discovery exclusion already says."""
bundle = candidate.get('bundle') or {}
account, model = bundle.get('account'), bundle.get('model')
said = {OUT_OF_AUTOMATIC_ROUTING: f'{host_policy.outside_routing(account)}: {host_policy.NOT_CHOSEN}',
UNCHECKED: {'default': auto_bundles.off_by_default(account, model), 'discovery': None}.get(
candidate.get('unchecked_by'), f'{host_policy.unchecked_routing(account, model)}: {host_policy.NOT_CHOSEN}')}
return [said.get(reason, reason) for reason in candidate.get('excluded') or () if said.get(reason, reason) is not None]
def local_settings(candidate):
"""The kinds of the user's local settings that exclude `candidate` when they are all that excludes it, else None."""
kinds = candidate.get('local_settings') or []
return kinds if kinds and len(kinds) == len(candidate.get('excluded') or ()) else None
def binding_wait(available_at, waiting_on, hours, window, waits):
"""`(available_at, waiting_on)` once `window` also holds the candidate back for `hours`: a candidate held by several
windows can start when the last of them returns, so the later reset sets the wait and its window is the one named.
`waits` (window -> hours) keeps every window that holds it back, for what reaches past some of them."""
waits[window] = hours
return (hours, window) if hours >= (available_at or 0) else (available_at, waiting_on)
def pin_matched_ids(task, ctx):
"""Bundle ids in `ctx['bundles']` named by `task`'s model/bundle constraint over `account_fleet.pin_pool` (every bundle
existing on the addressed accounts, before any filter of preference, availability, discovery state, modality or effort,
which `evaluate_bundle` applies to the named bundles only)."""
accounts = ctx['accounts']
cb = task.get('constraint_bundle')
pool = account_fleet.pin_pool(accounts, bundle_id=cb, source=(ctx.get('bundles') or {}).values())
if cb and not task.get('constraint_raw_model'):
return {(b.get('id') or b.get('bundle_id')) for b in pool if (b.get('id') or b.get('bundle_id')) == cb}
pin = task.get('constraint_raw_model') or task.get('constraint_model')
if not pin:
return set()
return {(b.get('id') or b.get('bundle_id')) for b in account_fleet.pin_matching_bundles(pin, pool, accounts)}
def model_pin_matches(task, bundle, accounts, pool=None):
"""Does the task's named model (or bundle id) name this bundle in `pool`? Reads `account_fleet.pin_matching_bundles`."""
cb = task.get('constraint_bundle')
if cb:
bid = (bundle or {}).get('id') or (bundle or {}).get('bundle_id')
if bid != cb:
return False
if not task.get('constraint_raw_model'):
return True
pin = task.get('constraint_raw_model') or task.get('constraint_model')
if not pin:
return False
if pool is None and isinstance(accounts, dict) and isinstance(accounts.get('accounts'), dict):
pool = account_fleet.pin_pool(accounts, bundle_id=cb, extra=(bundle,))
return account_fleet.pin_names_bundle(pin, bundle, accounts, pool=pool)
def exact_pin(task, bundle, accounts, pool=None):
"""The user named this bundle, or its model and effort: it keeps that effort and is only warned about when slow."""
bid = (bundle or {}).get('id')
if bid and bid in (task.get('pinned_bundle'), task.get('constraint_bundle')):
return True
if bid and not task.get('constraint_bundle') and (task.get('constraint_raw_model') or task.get('constraint_model')) == bid:
return True
return bool(task.get('constraint_effort')) and model_pin_matches(task, bundle, accounts, pool=pool)
def bundle_named(task, bundle, accounts, pool=None):
return exact_pin(task, bundle, accounts, pool=pool) or model_pin_matches(task, bundle, accounts, pool=pool)
def has_fixed_effort(bundle, accounts):
"""`model:effort` and `model-effort` launch ids run at that level only."""
if (bundle or {}).get('effort_fixed'):
return True
launch_id = str((bundle or {}).get('launch_model') or (bundle or {}).get('model') or '')
known = accounts if isinstance(accounts, dict) and 'models' in accounts else {'models': {}}
return split_model_effort(launch_id)[1] is not None or auto_bundles.split_effort(launch_id, known)[1] is not None
def named_effort_bundles(task, ctx):
"""Rows for the effort the user's pin names on a model the account has no row at that level for, so the score decides
among them exactly as it does among written rows. The launch contract is what offers a level; each row is the
model's own carrier with a distinct id, so it never passes for the carrier itself."""
effort = task.get('constraint_effort')
if not effort or task.get('constraint_bundle'):
return {}
accounts, groups = ctx['accounts'], {}
pin = task.get('constraint_raw_model') or task.get('constraint_model')
full_pool = account_fleet.pin_pool(accounts, source=ctx['bundles'].values())
source = account_fleet.pin_matching_bundles(pin, full_pool, accounts) if pin else full_pool
known = accounts.get('models') or {}
for bundle in source:
model = account_fleet.row_key(bundle['model'], accounts)
group_key = model if model in known else (account_fleet.advertised_model(bundle).lower() or model)
groups.setdefault((bundle['account'], bundle.get('harness'), group_key, model), []).append(bundle)
found = {}
for (_, _, _, model), rows in groups.items():
written = {str(quota_economy.scoring_effort(r, account_fleet.model_spec(accounts, r['model'])) or '').lower() for r in rows}
enabled = [r for r in rows if (not r.get('disabled') or r.get('disabled_by_default'))
and not r.get('discovery_unavailable') and not has_fixed_effort(r, accounts)]
carriers = enabled or [r for r in rows if not has_fixed_effort(r, accounts)]
# Each level goes to the first row that offers it, so the answer does not depend on the order of the rows.
for carrier in carriers:
allowed = account_fleet.dispatch_allowed_efforts(carrier, accounts, model)
levels = [effort]
if (effort not in allowed) if allowed is not None else not task.get('constraint_model'):
levels = []
for level in levels:
if level in written or (isinstance(carrier.get('efforts'), list) and level not in carrier['efforts']):
continue
written.add(level)
row_id = f"{carrier['id']}@{level}"
found[row_id] = {**{k: v for k, v in carrier.items()}, 'id': row_id, 'effort': level,
'pin_carrier': carrier['id']}
return found
def cooling_reason(bundle, entries, accounts, now=None):
"""launch_failures.reason for this row or any entry recorded under another spelling of its model: one model has one
cooldown, and a family guess is its own model."""
found = launch_failures.reason(bundle, entries, now=now)
if found or not entries:
return found
own = account_fleet.row_key(bundle.get('model') or '', accounts)
prefix = launch_failures.key(bundle.get('account'), '')
for entry in entries:
if entry.startswith(prefix) and account_fleet.row_key(entry[len(prefix):], accounts) == own:
return launch_failures.reason({**bundle, 'model': entry[len(prefix):]}, entries, now=now)
return None
def evaluate_bundle(bundle, task, ctx):
policy, accounts = ctx['policy'], ctx['accounts']
spec = accounts['accounts'][bundle['account']]
# Tables read by the name's own resolution: `model` is its key, for the matching that follows.
model = account_fleet.canonical(bundle['model'], accounts)
info = account_fleet.model_spec(accounts, bundle['model'])
kind = task['kind']
base_hours = task['hours']
base_chain_hours = task['chain_hours']
result = {'id': bundle['id'], 'bundle': {k: bundle.get(k) for k in ('provider', 'harness', 'account', 'model', 'effort', 'modeId', 'launch_model', 'draws', 'pin_carrier')
if bundle.get(k) is not None},
'excluded': [], 'notes': [], 'components': {}, 'score': None}
five = ctx['five_axis']
now = ctx['now']
# Economy pool mode (round 16, owner policy: a window's price must not depend on the current task's flags): the pool is
# what an account can run at all, evaluated with the task's flags neutralised (`rank` builds that task). It keeps the
# account-level exclusions (host policy, checkbox, discovery, holds, unverified, data errors, weekly window) and drops
# the ones that belong to the task: local-only, outbound, file transfer, image input, a named constraint, the demanded
# categories, whether the remaining capacity fits THIS task, and the 5-hour windows.
pool = bool(task.get('_economy_pool'))
def exclude(reason):
result['excluded'].append(reason)
def setting_exclude(kind, reason):
"""An exclusion by one of the user's local settings (LIFTED_BY_NAMING, cwd_regex)."""
exclude(reason)
result.setdefault('local_settings', []).append(kind)
def task_exclude(reason):
"""An exclusion that belongs to the current task, not to the account: the ranking keeps it, the pool does not."""
if not pool:
result['excluded'].append(reason)
def short_exclude(reason):
"""An exclusion by a 5-hour window alone: the work cannot start now, but the account still has its long windows, so
the economy (which prices only long windows) keeps this candidate as a substitute and in its own ladder."""
if pool:
return
result['excluded'].append(reason)
result.setdefault('_five_short_reasons', []).append(reason)
legacy = spec.get('legacy_meter')
if isinstance(legacy, str) and legacy:
exclude(legacy)
execution_account = accounts_override.preference_account(bundle, bundle['account'], accounts)
cwd_spec = accounts['accounts'].get(execution_account) or spec
if spec.get('_legacy_disabled_flag') or cwd_spec.get('_legacy_disabled_flag'):
exclude('legacy disabled flag remains; toggle the account in the tray or rerun agentcompass setup')
# A resumed worker whose directory could not be read (roll.build_task) is matched against no directory.
if not task.get('cwd_unread') and not account_fleet.account_cwd_allowed(cwd_spec, task.get('cwd')):
setting_exclude('cwd_regex', cwd_policy.refusal(execution_account, cwd_spec['cwd_regex'], task.get('cwd')))
# The bundle a recorded user_override or an explicit resume launches is the user's naming, which the two routing
# preferences do not govern. Only dispatch_registry.prepare passes it, as a route field from the decision's own
# constraint (task_intake.ROUTE_FIELDS); a caller's task cannot. Left in, the preferences kept it out of the
# economy, so a named fast route had no fast variant to launch.
user_named = bool(task.get('user_named_bundle')) and task['user_named_bundle'] == bundle['id']
matched_ids = None
if not pool and (task.get('constraint_model') or task.get('constraint_bundle')):
matched_ids = ctx.get('_pin_matched_ids')
if matched_ids is None:
matched_ids = pin_matched_ids(task, {**ctx, 'bundles': ctx.get('bundles') or {bundle['id']: bundle}})
pin_hit = bool(matched_ids is not None and bundle['id'] in matched_ids)
exact_hit = (
bool(bundle.get('id') and bundle['id'] in (task.get('pinned_bundle'), task.get('constraint_bundle')))
or bool(bundle.get('id') and not task.get('constraint_bundle')
and (task.get('constraint_raw_model') or task.get('constraint_model')) == bundle['id'])
or (bool(task.get('constraint_effort')) and pin_hit)
)
if not user_named and not host_policy.account_allowed(ctx['host_policy'], bundle['account']):
setting_exclude('allowed_accounts', OUT_OF_AUTOMATIC_ROUTING)
# Off by default (superseded in public data) is not the user's checkbox: naming the model still reaches it.
if (not user_named and bundle.get('disabled')
and not (bundle.get('disabled_by_default') and (exact_hit or pin_hit))):
# `excluded` holds UNCHECKED for all three (dispatch_registry.prepare lifts it by value); only the user's
# checkbox is the user's setting.
if bundle.get('discovery_unavailable') or bundle.get('disabled_by_default'):
exclude(UNCHECKED)
result['unchecked_by'] = 'discovery' if bundle.get('discovery_unavailable') else 'default'
else:
setting_exclude('checkbox', UNCHECKED)
if bundle.get('discovery_unavailable'):
exclude(bundle['discovery_unavailable'] if isinstance(bundle['discovery_unavailable'], str)
else 'model not detected by discovery for 7+ days')
if bundle['id'] in ctx['unavailable']:
exclude('marked unavailable by the parent')
cooling = cooling_reason(bundle, ctx.get('launch_failures'), accounts)
if cooling:
exclude(cooling)
latched_model = model_latch.reason(bundle, ctx.get('model_latches'))
if latched_model:
exclude(latched_model)
retrying = provider_retry.reason(bundle, ctx.get('provider_retries'))
if retrying:
exclude(retrying)
unoffered = live_models.reason(bundle, ctx.get('live_models'), bundle['account'], spec)
if unoffered:
exclude(unoffered)
# Only subscription accounts join automatic routing. A checked metered (API) bundle is a last resort:
# explicit selection, task.metered_ok, or plan()'s fallback pass when no subscription can take the work.
# Unknown billing stays explicit selection only.
billing = accounts_override.manual_only_billing(bundle['account'], accounts)
if billing and not task['metered_ok'] and not (billing == 'metered' and task.get('metered_fallback')):
if (not pin_hit and task.get('pinned_bundle') != bundle['id']
and not any(o.get('account') == bundle['account'] for o in ctx['overrides'])):
exclude(METERED_LAST_RESORT if billing == 'metered'
else f'{billing} billing: explicit selection only')
if not spec.get('verified', True):
exclude('account not verified as launchable')
if info.get('advisor_only') and (
kind not in info.get('advisor_for', ())
or (task.get('profile') or {}).get('access') == 'write'
):
exclude('advisor only; cannot execute')
account_out = bool(result['excluded']) # out by the host, the checkbox, a hold or billing: not by this task
if info.get('excluded_when') == 'local_only' and task['local_only']:
by = task_intake.LOCAL_ONLY_BY.get(task.get('local_only_by'), "the caller marked this work local-only (needs.local_only)")
task_exclude(f'{by}: it must run on this machine (in-place edits, local tools, secrets or live accounts), and this '
'worker runs remotely')
if info.get('excluded_when') == 'local_only' and task.get('outbound_ok') is False:
task_exclude('the parent disallowed outbound work; this worker runs remotely')
# Harness input limits apply to every bundle on the harness, manual or auto (HARNESS_CAPABILITIES).
harness = bundle.get('harness') or account_fleet.bundle_harness(bundle, bundle.get('account'), bundle.get('id'))
if task.get('parent_host') == 'herdr' and (herdr_cannot := host_herdr.cannot_run(bundle)):
task_exclude(herdr_cannot)
if (info.get('text_only') or not account_fleet.harness_transfers_files(harness)) and task.get('file_transfer'):
task_exclude('this text-only runner cannot transfer or apply local files' if info.get('text_only')
else 'this runner cannot transfer or apply local files')
if task['needs_visual'] and not account_fleet.harness_accepts_visual(harness, model):
task_exclude(f'{harness} cannot accept image attachments for {model}, so it cannot take visual or GUI work')
if (task['constraint_model'] or task.get('constraint_bundle')) and not pin_hit:
task_exclude('explicit model constraint')
if task.get('constraint_effort') and str(quota_economy.scoring_effort(bundle, info) or '').lower() != task['constraint_effort']:
task_exclude('explicit effort constraint')
# Rule B: a demanded category with no data at all, no cited usd on a metered account, or unknown image_input modality
# when the task demands image input (needs_visual, or tool_operation 3), leaves automatic routing; naming the model (--model, or its
# bundle) still dispatches it, with the data error as a warning and its cost unpriced.
state = info.get('five_axis') or {'missing': list(axis_scores.FIVE_AXES), 'usd_status': 'missing'}
billed = not (spec.get('unmetered') or spec.get('unlimited'))
named = not pool and (pin_hit or task.get('pinned_bundle') == bundle['id'])
img_info = state.get('image_input') or {'status': 'missing'}
# The task reads images when the classifier says so (needs_visual, which task intake also sets for needs_gui) or when
# it demands tool_operation 3: a model that takes no image input cannot do either.
needs_image_input = (task['needs_visual']
or task['five_axis_demand'].get('tool_operation', 0) >= axis_scores.IMAGE_INPUT_LEVEL) and not pool
eff = quota_economy.scoring_effort(bundle, info)
values = axis_scores.five_axis_values(state, eff)
if needs_image_input and img_info.get('status') == 'no':
exclude(f"model does not accept image input ({img_info.get('source') or 'public data'})")
def _missing_targets(axis, level):
if level < five['demanded_level']:
return []
if axis in axis_scores.LEVEL_FACETS:
selected = axis_scores.LEVEL_FACETS[axis].get(level) or ()
if not selected:
return []
return [f'{axis_scores.FACET_CATEGORY[f]}.{f}' for f in selected
if (values.get(axis) or {}).get(f) is None]
if values.get(axis) is not None:
return []
cat_cell = (state.get('categories') or {}).get(axis) or {}
facets = cat_cell.get('facets') if isinstance(cat_cell, dict) else None
req = user_scores.CATEGORY_FACETS.get(axis) or ()
if isinstance(facets, dict) and len(req) > 1:
miss_f = [f'{axis}.{f}' for f in req if axis_scores._at_effort(facets.get(f), eff) is None]
if miss_f and len(miss_f) < len(req):
return miss_f
return [axis]
blind, missing_targets = [], []
if not pool:
for a, w in task['five_axis_demand'].items():
miss = _missing_targets(a, w)
if miss:
blind.append(a)
missing_targets.extend(miss)
if blind:
first_target = missing_targets[0]
cats_map = state.get('categories') or {}
has_high = any(
axis_scores._facet_at_effort(cats_map, *t.split('.', 1), 'high') is not None if '.' in t
else ((cats_map.get(t) or {}).get('value') is not None)
for t in missing_targets
)
eff_note = f' at effort {eff}' if eff and has_high else ''
blind_msg = (
f'no trusted data for the demanded category {", ".join(blind)} on {model}{eff_note} '
f'(missing {", ".join(missing_targets)}; fill with `agentcompass score set {model} {first_target} <level>` '
f'or in the tray Manual Scores panel)'
)
else:
blind_msg = None
if blind_msg and not pool:
result['_five_blind_msg'] = blind_msg
errors = (
([blind_msg] if blind_msg else [])
+ (['usd_per_hour has no cited source on a metered account'] if billed and state.get('usd_status') != 'ok' else [])
+ (['image_input modality has no cited source'] if needs_image_input and img_info.get('status') not in ('yes', 'no') else [])
)
for error in errors:
if named:
unpriced = ' (its cost is not priced)' if error.startswith('usd_per_hour') else ''
result['notes'].append(f'data error: {error}; dispatched because it was named{unpriced}')
result.setdefault('data_errors', []).append(error)
else:
exclude(f'data error: {error} (out of automatic routing; name the model to dispatch it)')
# What the overlay applied, read at the effort this candidate is scored with (user_scores owns the text).
manual_applied = user_scores.applied_scores(state, axis_scores.reading_effort(state, eff))
if manual_applied:
result['notes'].append(user_scores.applied_note(manual_applied, axis_scores.reading_effort(state, eff)))
result['manual_scores'] = manual_applied
estimated = state.get('estimates') or {}
if estimated:
result['notes'].append('facets estimated (same-series sibling or the fit on the tier; no board of their own): '
+ ', '.join(sorted(estimated)))
result['estimated_axes'] = sorted({name.split('.')[0] for name in estimated})
# One basis for the time a task takes, its spend and the window it needs (`duration_at`): Time/Task at this effort. Where
# the data has none, the candidate leaves automatic routing; a named dispatch carries the task's own hours estimate
# unscaled and the data error, and nothing stands in for the model's pace.
dur_mult, dur_basis = quota_economy.duration_at(info, eff)
if dur_mult is None:
error = (f'no Time/Task for {model} at {eff or "its single configuration"} (Artificial Analysis, arena.ai Agent on '
f'its scale, or a same-series effort conversion)')
if named:
result['notes'].append(f'data error: {error}; dispatched because it was named, hours are the task\'s own estimate')
result.setdefault('data_errors', []).append(error)
else:
exclude(f'data error: {error} (out of automatic routing at this effort; name the model to dispatch it)')
hours = base_hours if dur_mult is None else base_hours * dur_mult
chain_hours = base_chain_hours if dur_mult is None else base_chain_hours * dur_mult
result['components']['duration_source'] = (dur_basis or {}).get('method') or 'unknown'
if dur_mult is not None:
result['notes'].append(f'time at {eff or "its single configuration"}: {hours:.2f}h from '
f'{(dur_basis or {}).get("method")} ({(dur_basis or {}).get("source") or "series"})')
expires = quota_pools.parse_time(spec.get('expires_at'))
if expires is not None and (expires - now).total_seconds() < hours * 3600:
task_exclude('subscription ends before the work would finish')
worker = task['worker']
worker_bundle = worker.get('bundle') if worker else None
worker_is_fast = bool(worker and (worker.get('fast') or (isinstance(worker_bundle, str) and worker_bundle.endswith(fast_modes.ID_SUFFIX))))
worker_base = (worker_bundle[:-len(fast_modes.ID_SUFFIX)]
if isinstance(worker_bundle, str) and worker_bundle.endswith(fast_modes.ID_SUFFIX)
else worker_bundle)
same_worker_base = (
bool(worker)
and worker_base == bundle['id']
and not (task.get('read_only_review') and worker.get('read_only_review') is False)
)
resumes = same_worker_base and not worker_is_fast
worn = worker_sessions.worker_worn(worker, policy)
context_extra = 0.0
if resumes:
carried = worker_sessions.worker_historical_tokens(worker)
if not worn:
limit = worker_sessions.rotation_threshold(policy)
context_extra = policy['context_burn_extra_at_250k'] * quota_pools.clamp(carried / limit, 0, 1)
if worker.get('telemetry_status') == 'partial':
result['notes'].append('session telemetry partial; carried tokens are a lower bound')
if worker.get('compacted_tokens') is None and worker.get('compactions') is None and worker.get('historical_context_tokens') is None:
result['notes'].append('compaction history unknown (give chain.worker.agent_id); resuming')
elif worn:
result['notes'].append(f"the worker has carried {carried / 1e6:.1f}M historical tokens of context over "
f"{worker.get('compactions', '?')} compactions; rotate with a handoff packet")
context_burn = 1 + context_extra
# `usd_per_hour` is the model's dollars per hour of its own work (a rate), so the spend is usd/h x the hours this
# candidate takes at its effort, and the window's quota rate carries no second effort factor.
econ_hours, econ_chain_hours = hours, chain_hours
step_usd = quota_economy.bundle_task_usd(spec, model, econ_hours, info, context_burn)
chain_usd = quota_economy.bundle_task_usd(spec, model, econ_chain_hours, info, context_burn)
available_at = None
waiting_on = None # the window whose reset sets available_at
waits = {} # every window that holds the candidate back (binding_wait)
five_window_states = {}
long_applicable = False # the bundle draws at least one long reset-period window (read or not)
if spec.get('unlimited'):
result['notes'].append('unlimited usage')
acct_overrides = [o for o in ctx.get('overrides', ()) if o['account'] == bundle['account']]
acct_pool = (account_fleet.pin_pool(accounts, account=bundle['account'],
source=(ctx.get('bundles') or {}).values(), extra=(bundle,))
if acct_overrides else None)
def _override_matches(o):
if not o.get('model'):
return True
if model_pin_problem(o['model']) is not None:
return False
return account_fleet.pin_names_bundle(o['model'], bundle, accounts, pool=acct_pool)
boost = sum(float(o.get('boost') or 0) for o in acct_overrides
if _override_matches(o)
and (quota_overrides.expected_reset_time(o) is None or quota_overrides.expected_reset_time(o) <= now))
for intent in acct_overrides:
if intent.get('expected_reset_at') and _override_matches(intent):
result['notes'].append(f"expected reset {intent['expected_reset_at']}: {intent['reset_expectation_status']}; capacity requires a fresh observation")
if boost:
result['components']['boost'] = round(boost, 3)
result['notes'].append(f'temporary boost {boost:+.2f} from a user override')
# Purpose-limited unmetered channels (Dot, ChatGPT Web) have no quota to relieve: never another window's substitute.
result['_five_unmetered'] = bool(spec.get('unmetered') or spec.get('unlimited'))
# A metered (API) account is a last resort: it has a ladder of its own but is never another window's substitute.
result['_five_metered'] = billing == 'metered'
if not spec.get('unmetered'):
acct_usage = ctx['usage'].get('accounts', {}).get(bundle['account']) or {}
acct_stale = acct_usage.get('status') == 'stale' or acct_usage.get('stale') is True
latched = quota_pools.latched_windows(accounts, bundle['account'])
named = not pool and (pin_hit or task.get('pinned_bundle') == bundle['id'])
for wid, wspec, observed in quota_pools.applicable_windows(spec, model, acct_usage, bundle, accounts):
result.setdefault('_five_axis_windows', []).append(wid)
long_applicable = long_applicable or not quota_pools.window_is_short(wspec, policy)
observed = quota_pools.cycle_observation(wspec, observed, acct_usage, now, read_failed_ok=True)
refusal = quota_pools.window_refusal(wid, wspec, observed, acct_usage, now, latched)
if refusal:
(short_exclude if quota_pools.window_is_short(wspec, policy) else exclude)(refusal)
expected = quota_overrides.window_reset_expectation(
ctx.get('overrides', ()), bundle['account'], model, wid, spec, accounts, now,
bundle=bundle, pool=acct_pool,
)
state = quota_pools.window_state(wid, wspec, observed, spec, now, policy['short_window_hours'],
account_stale=acct_stale, expected_reset_at=expected)
unit = quota_economy.unit_rate(spec, wid, model, info)
cap_usd_w = quota_economy.capacity_usd(spec, wid)
if cap_usd_w is None:
if state['known'] and state['remaining'] <= policy['safety_margin_pct']:
exclude(f'{wid} has {state["remaining"]:.0f}% left, inside the {policy["safety_margin_pct"]}% reserve '
'(capacity unknown, so the need cannot be priced)')
result['wait_until_hours'] = max(result.get('wait_until_hours', 0), state['reset_hours'])
continue
err = f'{wid} capacity_unknown (quota unknown, spend unpriced)'
plan_suffix = f"; plan unknown ({spec['plan_unknown_reason']})" if spec.get('plan_unknown_reason') else ''
if named:
local_u = spec.get('local_spend_usd')
local_note = f'; local opencode.db lower-bound spend ~${local_u:.2f}' if isinstance(local_u, (int, float)) and local_u > 0 else ''
result['notes'].append(f'data error: {err}; dispatched because it was named{plan_suffix}{local_note}')
result.setdefault('data_errors', []).append(f'{err}{plan_suffix}')
else:
exclude(f'data error: {err}{plan_suffix} (out of automatic routing; name the model to dispatch it)')
continue
if not state['known']:
if state['short']:
w_len = float(wspec.get('hours') or 0) or econ_hours
step = None if unit is None else min(econ_hours, w_len) * unit * context_burn
margin_pct = quota_pools.short_window_limits(spec, wid, policy, now)[2]
if step is not None and 100.0 - step < margin_pct:
short_exclude(f'{wid} full capacity (100%) is short of the ~{step:.1f}% the work needs plus {margin_pct}% margin')
elif (observed and (reset_dt := quota_pools.parse_time(observed.get('resets_at'))) is not None
and reset_dt > now and observed.get('reset_time_passed') is not True):
wait = (reset_dt - now).total_seconds() / 3600
short_exclude(f'{wid} has an active reset in {wait:.1f}h with unknown remaining percentage')
continue
state, reserved_usd, reserved_pct = quota_pools.reserve_inflight(
bundle['account'], spec, wid, wspec, state, ctx.get('records'), accounts, policy, now)
if reserved_usd > 0:
result['notes'].append(f'{wid}: -${reserved_usd:.2f} (-{reserved_pct:.1f}%) reserved by in-flight {bundle["account"]} dispatch')
if state['short']:
w_len = float(wspec.get('hours') or 0) or econ_hours
tau = state.get('reset_hours', w_len)
h_before = min(econ_hours, w_len) if (tau is None or tau == float('inf') or state.get('unstarted')) else min(econ_hours, max(0.0, float(tau)))
step = None if unit is None else h_before * unit * context_burn
step_period = None if unit is None else min(max(0.0, econ_hours - h_before), w_len) * unit * context_burn
small_short, floor_pct, margin_pct = quota_pools.short_window_limits(spec, wid, policy, now)
empty = state['remaining'] <= floor_pct
if empty or (step is not None and state['remaining'] - step < margin_pct):
wait = state['reset_hours']
what = 'is empty' if empty else f'has {state["remaining"]:.0f}%, short of the ~{step:.1f}% the work needs,'
post_period = None if unit is None else min(econ_hours, w_len) * unit * context_burn
if post_period is not None and 100.0 - post_period < margin_pct:
short_exclude(f'{wid} full capacity (100%) is short of the ~{post_period:.1f}% the work needs plus {margin_pct}% margin')
elif (not small_short and not state['stale'] and 0 < wait <= policy['defer_if_reset_within_hours']
and wait >= state.get('natural_reset_hours', wait)):
available_at, waiting_on = binding_wait(available_at, waiting_on, wait, wid, waits)
result['notes'].append(f'{wid} {what} for {wait:.1f}h; start after its reset')
renewed = quota_pools.after_reset(state, w_len)
if renewed is not None:
renewed['unstarted'] = True
five_window_states[wid] = (wspec, renewed, cap_usd_w, unit)
else:
short_exclude(f'{wid} {what} for {wait:.1f}h')
continue
if step_period is not None and 100.0 - step_period < margin_pct:
short_exclude(f'{wid} full capacity (100%) is short of the ~{step_period:.1f}% the work needs plus {margin_pct}% margin')
continue
five_window_states[wid] = (wspec, state, cap_usd_w, unit)
continue
if unit is None:
continue
w_len = wspec.get('hours')
tau = state.get('reset_hours')
req_hours = econ_chain_hours if task['coupling'] == 'high' else econ_hours
h_before = req_hours if (tau is None or tau == float('inf') or w_len is None) else min(req_hours, max(0.0, float(tau)))
need = h_before * unit * context_burn
if (not pool and state['remaining'] - need < policy['safety_margin_pct'] and w_len is not None
and state['reset_hours'] <= policy['defer_if_reset_within_hours']):
renewed = quota_pools.after_reset(state, w_len)
if renewed is not None and renewed['expiry_hours'] is not None and renewed['expiry_hours'] < hours:
renewed = None
if renewed is not None and 100.0 - min(req_hours, float(w_len)) * unit * context_burn >= policy['safety_margin_pct']:
state = renewed
available_at, waiting_on = binding_wait(available_at, waiting_on, renewed['deferred_hours'], wid,
waits)
result['_five_long_wait'] = True # a long window must renew first: no absorber of other work now
need = min(req_hours, float(w_len)) * unit * context_burn
result['notes'].append(f'{wid} renews in {renewed["deferred_hours"]:.1f}h; start then (assumes a full renewal)')
if state['remaining'] - need < policy['safety_margin_pct']:
if state['reset_hours'] == float('inf'):
task_exclude(f'{wid} has {state["remaining"]:.0f}% left; the work needs ~{need:.1f}% plus a {policy["safety_margin_pct"]}% margin '
'(starter allowance does not reset)')
else:
task_exclude(f'{wid} has {state["remaining"]:.0f}% left; the work needs ~{need:.1f}% plus a {policy["safety_margin_pct"]}% margin '
f'and it resets in {state["reset_hours"] / 24:.1f} days')
result['wait_until_hours'] = max(result.get('wait_until_hours', 0), state['reset_hours'])
elif w_len is not None and req_hours > h_before:
rem_need = min(req_hours - h_before, float(w_len)) * unit * context_burn
if 100.0 - rem_need < policy['safety_margin_pct']:
task_exclude(f'{wid} full capacity (100%) is short of the ~{rem_need:.1f}% the work needs per period plus {policy["safety_margin_pct"]}% margin')
five_window_states[wid] = (wspec, state, cap_usd_w, unit)
cap_ev = ((spec.get('capacity_source') or {}).get(wid) or {}).get('evidence')
if cap_ev:
result['notes'].append(f'{wid} capacity: {cap_ev}')
result['components'].setdefault('capacity_evidence', {})[wid] = cap_ev
if state['stale'] and 'capacity from a stale observation' not in result['notes']:
result['notes'].append('capacity from a stale observation')
if acct_usage.get('status') != 'fresh' and acct_usage.get('fetched_at') and not result['components'].get('reading_at'):
# Priced from the last successful reading of this reset period (owner decision of round 13b): labelled
# with its read time so plan, roll and the tray say how old the quota figure is.
result['components']['reading_at'] = acct_usage['fetched_at']
result['notes'].append(f"{wid}: priced from the reading taken at {acct_usage['fetched_at']} "
f"(the latest read did not succeed; it is the last good reading of this reset period)")
if spec.get('capacity_unknown') and not result.get('_five_axis_windows'):
err = 'capacity_unknown (quota unknown, spend unpriced)'
plan_suffix = f"; plan unknown ({spec['plan_unknown_reason']})" if spec.get('plan_unknown_reason') else ''
if named:
local_u = spec.get('local_spend_usd')
local_note = f'; local opencode.db lower-bound spend ~${local_u:.2f}' if isinstance(local_u, (int, float)) and local_u > 0 else ''
result['notes'].append(f'data error: {err}; dispatched because it was named{plan_suffix}{local_note}')
result.setdefault('data_errors', []).append(f'{err}{plan_suffix}')
else:
exclude(f'data error: {err}{plan_suffix} (out of automatic routing; name the model to dispatch it)')
if quota_pools.metered_usd_uncapped(spec):
pass # no budget: not a window, not excluded for exhaustion, still a metered last resort
elif (not spec.get('unlimited') and not account_out and not result.get('data_errors')
and not any(not st.get('short') for _, st, _, _ in five_window_states.values())):
# Priced on the weekly window or not at all (round 15): a candidate with no long reset-period window in its
# state cannot be priced and cannot stand in for another account's work, so it leaves automatic routing as
# a surfaced data error. Three causes, three messages: nothing of the account can be read (round 13); the
# weekly window is declared but not read while the 5-hour window is; no weekly window exists at all.
if not result.get('_five_axis_windows') or not long_applicable:
err = QUOTA_NO_LONG_WINDOW if not long_applicable else QUOTA_READING_UNAVAILABLE
elif five_window_states:
err = QUOTA_LONG_UNREAD
else:
err = QUOTA_READING_UNAVAILABLE
if named:
result['notes'].append(f'data error: {err}; dispatched because it was named')
result.setdefault('data_errors', []).append(err)
else:
exclude(f'data error: {err} (out of automatic routing; name the model to dispatch it)')
result['_five_window_states'] = five_window_states
result['_five_econ_hours'] = econ_hours
result['_five_chain_hours'] = econ_chain_hours
result['_five_context_burn'] = context_burn
if not five_window_states and not result['excluded'] and not result.get('data_errors'):
result['notes'].append('no current capacity observation; unknown is not exhausted')
# A declared dispatch cadence holds every account to it. It is priced only where it is the account's whole quota (an
# unmetered account); beside quota windows it is a gate, and those windows price the task as they do without it.
cad_got = quota_pools.cadence_window_state(bundle['account'], spec, ctx.get('records'), now)
if cad_got is not None:
wspec, state, cap_count = cad_got
need_pct = 100.0 / cap_count
if not pool and state['remaining'] + 1e-9 < need_pct:
if state['reset_hours'] <= policy['defer_if_reset_within_hours']:
renewed = quota_pools.after_reset(state, wspec['hours'])
if renewed is not None:
state = renewed
available_at, waiting_on = binding_wait(available_at, waiting_on, renewed['deferred_hours'], CADENCE,
waits)
result['_five_long_wait'] = True
result['notes'].append(
f'cadence frees {renewed["remaining"] * cap_count / 100.0:g} of {cap_count:g} dispatches in '
f'{renewed["deferred_hours"]:.1f}h; start then'
)
if state['remaining'] + 1e-9 < need_pct:
task_exclude(OVER_CADENCE)
result['notes'].append(f'cadence has 0 of {cap_count:g} dispatches left; one frees in '
f'{state["reset_hours"]:.1f}h')
result['wait_until_hours'] = max(result.get('wait_until_hours', 0), state['reset_hours'])
if spec.get('unmetered') and not spec.get('unlimited'):
result.setdefault('_five_axis_windows', []).append('cadence')
five_window_states['cadence'] = (wspec, state, cap_count, None)
result['_five_window_states'] = five_window_states
result['_five_window_spend'] = 1.0
result['_five_cadence_instant'] = True
result['_five_econ_hours'] = econ_hours
result['_five_chain_hours'] = econ_chain_hours
result['_five_context_burn'] = context_burn
if (pool or (not task.get('disable_fast') and available_at is None)) and not result['excluded'] and not result.get('_five_long_wait'):
fast_info = ((info.get('five_axis') or {}).get('fast_mode')) or {}
if fast_info.get('status') == 'ok' and fast_modes.supported(harness, model, info):
f_price = float(fast_info.get('price_multiplier') or 2.0)
f_speed = float(fast_info.get('speed_multiplier') or 1.0)
f_econ_h = econ_hours / f_speed if f_speed > 0 else econ_hours
f_chain_h = econ_chain_hours / f_speed if f_speed > 0 else econ_chain_hours
f_hours = hours / f_speed if f_speed > 0 else hours
f_step_usd = step_usd * f_price
f_chain_usd = chain_usd * f_price
# No window to check (a metered account without a budget, an unlimited one): fast is offered only to a
# route that names it, which is that route whatever prices it.
fast_ok = bool(five_window_states) or bool(named and task.get('force_fast'))
for wid, (wspec, st, cap_usd_w, unit) in five_window_states.items():
if pool and st.get('short'):
continue
if unit is None:
fast_ok = False
break
if pool:
continue # the pool asks what the account can run, not whether it fits the remaining capacity now
f_unit = unit * f_price * f_speed * context_burn # per fast hour; x f_econ_h is price x the normal spend
w_len = wspec.get('hours')
tau = st.get('reset_hours')
if st['short']:
w_l = float(w_len or 0) or f_econ_h
h_bef = min(f_econ_h, w_l) if (tau is None or tau == float('inf') or st.get('unstarted')) else min(f_econ_h, max(0.0, float(tau)))
f_step = h_bef * f_unit
f_per = min(max(0.0, f_econ_h - h_bef), w_l) * f_unit
_, fl_pct, mg_pct = quota_pools.short_window_limits(spec, wid, policy, now)
if st['remaining'] <= fl_pct or st['remaining'] - f_step < mg_pct or 100.0 - f_per < mg_pct:
fast_ok = False
break
else:
req_h = f_chain_h if task['coupling'] == 'high' else f_econ_h
h_bef = req_h if (tau is None or tau == float('inf') or w_len is None) else min(req_h, max(0.0, float(tau)))
f_need = h_bef * f_unit
if st['remaining'] - f_need < policy['safety_margin_pct']:
fast_ok = False
break
if w_len is not None and req_h > h_bef:
f_rem = min(req_h - h_bef, float(w_len)) * f_unit
if 100.0 - f_rem < policy['safety_margin_pct']:
fast_ok = False
break
if task.get('force_fast') and not fast_ok:
exclude(f'fast mode ({f_price:g}x burn) exceeds remaining window safety margin')
elif not fast_ok:
result['fast_not_offered'] = f'fast mode ({f_price:g}x burn) exceeds the remaining window safety margin'
elif fast_ok:
result['_five_fast_candidate'] = {
'price_multiplier': f_price, 'speed_multiplier': f_speed,
'econ_hours': f_econ_h, 'chain_hours': f_chain_h, 'hours': f_hours,
'step_usd': f_step_usd, 'chain_usd': f_chain_usd, 'source': fast_info.get('source'),
}
elif task.get('force_fast') and not fast_modes.supported(harness, model, info):
# Whether the harness can launch this model fast is a fact (fast_modes), the same answer `record` gives.
exclude(fast_modes.refusal(harness, model, info))
elif not fast_modes.supported(harness, model, info):
result['fast_not_offered'] = fast_modes.refusal(harness, model, info)
elif not task.get('force_fast'):
result['fast_not_offered'] = fast_modes.price_gap(model, info)
elif task.get('force_fast'):
err = fast_modes.price_gap(model, info)
if named:
# The contract exists and only a price datum is missing: a named fast route launches, like any other
# unpriced named route, reserved at the plain mode's spend (the fast burn is not published).
result['notes'].append(f'data error: {err}; dispatched because it was named, its fast-mode cost is not '
"priced (reserved at the plain mode's spend)")
result.setdefault('data_errors', []).append(err)
result['_five_fast_candidate'] = {
'price_multiplier': None, 'speed_multiplier': None, 'unpriced': True,
'econ_hours': econ_hours, 'chain_hours': econ_chain_hours, 'hours': hours,
'step_usd': step_usd, 'chain_usd': chain_usd, 'source': None,
}
else:
result['notes'].append(f'data error: {err}; fast mode not enabled')
result.setdefault('data_errors', []).append(err)
if not result['excluded'] and not result.get('_five_fast_candidate') and 'fast_not_offered' not in result:
result['fast_not_offered'] = ('fast mode is switched off for this task' if task.get('disable_fast') else
'this candidate must wait for a window reset, and fast mode is offered only to one '
'that can start now' if available_at is not None else None)
result['capacity'] = None # the economy sets the priced window (apply_five_axis_economy)
result['expected_hours'] = round(hours, 2)
result['duration_hours'] = round(hours, 4)
if dur_mult is not None:
result['components']['duration_multiplier'] = round(dur_mult, 4)
result['expected_pct'] = result['chain_pct'] = None
result['expected_usd'] = None if billing == 'unknown' else round(step_usd, 2)
result['chain_usd'] = None if billing == 'unknown' else round(chain_usd, 2)
result['available_in_hours'] = available_at
result['waiting_on'] = waiting_on if available_at else None
if waits:
result['waits'] = waits # what dispatch_registry.prepare reaches past for a named route (AUTOMATIC_ROUTING_WAITS)
action = 'start'
if worker:
if worn:
action = 'rotate'
else:
action = 'resume' if resumes else 'switch'
result['action'] = action
# Quality is the continuous fit of the category scores at this effort to the demand, and nothing else (the effort's
# extra burn is priced in its usd and its gain in the categories' effort shift). The economy decides.
fit, axis_rows = axis_scores.five_axis_fit(values, task['five_axis_demand'], task['five_axis_targets'])
result['_five_values'] = values # the economy re-reads this candidate's scores at its effort over the demand grid
# The candidate's four category scores at this effort: what the radar and the explanation show (`aesthetics` and
# `tool_operation`: the weaker of their facets that have a value).
shown = {}
for a in axis_scores.FIVE_AXES:
if a in axis_scores.LEVEL_FACETS:
present = [z for z in (values.get(a) or {}).values() if z is not None]
shown[a] = min(present) if present else None
else:
shown[a] = values.get(a)
result['components'].update({a: None if v is None else round(v, 4) for a, v in shown.items()})
result['components'].update({f'share_{a}': v for a, v in
axis_scores.five_axis_shares(values, ctx['accounts']['_five_axis']['targets']).items()})
result['components'].update(five_axis_level=round(axis_scores.five_axis_level(axis_rows), 4),
five_axis_shortfall=round(sum(r['demand'] * r['shortfall'] for r in axis_rows.values())
/ max(1, sum(r['demand'] for r in axis_rows.values())), 4))
result['five_axis_demand'] = task['five_axis_demand']
result['five_axis_fit'] = axis_rows
headroom = axis_scores.five_axis_headroom(axis_rows, ctx['accounts']['_five_axis']['targets'])
if headroom:
result['five_axis_headroom'] = headroom # level-3 credit above the anchor, for the decision radar's hover
usd_rate = 0.0 if spec.get('unlimited') or spec.get('unmetered') or billing == 'unknown' else quota_economy.model_usd_per_hour(model, info) # dollars per hour of this candidate's own work; effort is in its duration
result['five_axis_price'] = {'usd_per_hour': round(usd_rate, 4)}
result['components'].update({'fit': round(fit, 3), 'continuity': 0.0})
if action == 'resume' and task['coupling'] in ('medium', 'high'):
# a related task (--resume or an auto-matched continuation of the same chain) continues its worker: one demand level's worth
result['components']['continuity'] = round(task['five_axis_continuity'], 4)
return result
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |