Skip to content

akiiwa-games/AgentCompass / skills/agent-compass

agent-compass

Flagged

Choose owner, model and effort, outbound authorization, advisor use, and continuation for delegated work. Reads local usage and effective account policy, profiles the task with the configured System-One scorer, then allocates a compatible worker. Use for independent worker or advisor work; ordinary local execution needs no routing.

FileKindSize
SKILL.mdskill md5.5 KB
agents/openai.yamlconfig427 B
calibration/README.mdmarkdown55.2 KB
calibration/arena_capture.pyscript14.0 KB
calibration/community.pyscript25.2 KB
calibration/five_axis.pyscript196.4 KB
calibration/measurements.pyscript68.4 KB
calibration/refresh.pyscript48.6 KB
calibration/series.pyscript10.2 KB
calibration/sources.pyscript24.1 KB
calibration/tiers.pyscript14.6 KB
grok-bot/.gitignoreother47 B
grok-bot/dependency_patch.pyscript4.5 KB
grok-bot/install_cli.pyscript3.6 KB
grok-bot/test-recovery.mjsscript8.6 KB
grok-bot/test_install_cli.pyscript4.1 KB
grok-bot/upstream.jsonconfig3.4 KB
references/accounts.jsonconfig155.9 KB
references/classifier-cases.jsonconfig21.5 KB
references/dispatch-lifecycle.mdmarkdown28.9 KB
references/dispatch-rules.mdmarkdown11.3 KB
references/effort-matching.mdmarkdown5.1 KB
references/fast-modes.mdmarkdown3.5 KB
references/five-axis-scores.jsonconfig3.1 MB
references/gemini-decisions-api.mdmarkdown2.9 KB
references/grok-bot.mdmarkdown13.0 KB
references/herdr.mdmarkdown5.1 KB
references/jev-api.mdmarkdown1.6 KB
references/level-3-conversion.jsonconfig2.4 KB
references/model-capabilities.jsonconfig3.6 KB
references/model-capabilities.mdmarkdown14.6 KB
references/parent-address-gate.mdmarkdown10.1 KB
references/plan-templates.jsonconfig13.2 KB
references/provider-discovery.mdmarkdown11.5 KB
references/provider-usage.mdmarkdown4.1 KB
references/public-data.jsonconfig816 B
references/public-measurements.jsonconfig1.3 MB
references/roll-reference.mdmarkdown26.7 KB
references/router-internals.mdmarkdown93.7 KB
references/task-classifier.mdmarkdown6.1 KB
references/user-scores.jsonconfig6.8 KB
references/work-kinds.mdmarkdown4.7 KB
references/worker-communication.mdmarkdown2.1 KB
scripts/account_fleet.pyscript32.9 KB
scripts/account_state.pyscript22.2 KB
scripts/accounts_override.pyscript14.8 KB
scripts/allocator.pyscript19.8 KB
scripts/antigravity_plan.pyscript18.7 KB
scripts/antigravity_telemetry.pyscript5.1 KB
scripts/auto_bundles.pyscript31.3 KB
scripts/axis_scores.pyscript40.3 KB
scripts/caller_input.pyscript3.0 KB
scripts/candidate_eval.pyscript54.0 KB
scripts/candidate_ranking.pyscript9.5 KB
scripts/capability_catalog.pyscript8.0 KB
scripts/check_router_contract.pyscript2.8 KB
scripts/conftest.pyscript2.5 KB
scripts/cwd_policy.pyscript1.5 KB
scripts/data_age.pyscript1.5 KB
scripts/decision_ledger.pyscript18.9 KB
scripts/demo_plan.pyscript6.3 KB
scripts/dispatch_fallback.pyscript40.3 KB
scripts/dispatch_flow.pyscript58.7 KB
scripts/dispatch_ground_truth.pyscript43.5 KB
scripts/dispatch_registry.pyscript113.7 KB
scripts/entry.pyscript1.7 KB
scripts/env_vars.pyscript580 B
scripts/fast_modes.pyscript2.5 KB
scripts/fixtures/grok-api-responses.jsonconfig907 B
scripts/fixtures/reference-accounts.jsonconfig26.9 KB
scripts/grok_bot_paths.pyscript1005 B
scripts/host_herdr.pyscript16.8 KB
scripts/host_placement.pyscript9.4 KB
scripts/host_policy.pyscript10.1 KB
scripts/launch_failures.pyscript5.9 KB
scripts/launch_values.pyscript12.3 KB
scripts/ledger_snapshot.pyscript11.1 KB
scripts/live_models.pyscript21.1 KB
scripts/model_latch.pyscript7.6 KB
scripts/model_resolution.pyscript18.0 KB
scripts/override_store.pyscript2.2 KB
scripts/plan_state_residue.pyscript5.0 KB
scripts/provider_inventory.pyscript128.2 KB
scripts/provider_retry.pyscript34.6 KB
scripts/public_data.pyscript59.2 KB
scripts/quota_economy.pyscript38.0 KB
scripts/quota_exhaustion.pyscript5.9 KB
scripts/quota_overrides.pyscript25.6 KB
scripts/quota_pools.pyscript29.8 KB
scripts/reference_provenance.pyscript32.3 KB
scripts/released_names.pyscript3.8 KB
scripts/roll.pyscript106.3 KB
scripts/route_judge.pyscript83.4 KB
scripts/route_simulation.pyscript26.3 KB
scripts/router_test_support.pyscript6.7 KB
scripts/run_grok_bot.pyscript18.8 KB
scripts/scorer_client.pyscript55.2 KB
scripts/stage_help.pyscript5.8 KB
scripts/state_upgrade.pyscript24.0 KB
scripts/subscription_cli.pyscript3.6 KB
scripts/task_intake.pyscript33.6 KB
scripts/test_account_identity.pyscript7.1 KB
scripts/test_account_state.pyscript63.3 KB
scripts/test_accounts_override.pyscript30.1 KB
scripts/test_allocator.pyscript94.2 KB
scripts/test_antigravity_plan.pyscript13.2 KB
scripts/test_applicable_quota.pyscript9.1 KB
scripts/test_auto_bundles.pyscript10.8 KB
scripts/test_billing_routing_policy.pyscript9.9 KB
scripts/test_cadence_count.pyscript32.2 KB
scripts/test_calibration_units.pyscript27.0 KB
scripts/test_capability_catalog.pyscript5.0 KB
scripts/test_decision_ledger.pyscript8.5 KB
scripts/test_demo_plan.pyscript3.4 KB
scripts/test_derived_state.pyscript21.6 KB
scripts/test_dispatch.pyscript322.3 KB
scripts/test_dispatch_answers.pyscript14.8 KB
scripts/test_dispatch_fallback.pyscript53.2 KB
scripts/test_entry.pyscript1.6 KB
scripts/test_fast_modes.pyscript9.9 KB
scripts/test_five_axis.pyscript290.1 KB
scripts/test_five_axis_economy.pyscript105.0 KB
scripts/test_fleet_parity.pyscript15.6 KB
scripts/test_grok_bot.pyscript47.7 KB
scripts/test_ground_truth_recording.pyscript68.8 KB
scripts/test_herdr.pyscript14.1 KB
scripts/test_host_policy.pyscript24.6 KB
scripts/test_launch_values.pyscript8.0 KB
scripts/test_metered_usd.pyscript60.4 KB
scripts/test_model_resolution.pyscript9.2 KB
scripts/test_named_route_facts.pyscript28.6 KB
scripts/test_native_hosts.pyscript6.0 KB
scripts/test_new_user_fleet.pyscript3.3 KB
scripts/test_no_hardcode_classification.pyscript75.1 KB
scripts/test_no_legacy_scoring.pyscript6.5 KB
scripts/test_override_store.pyscript2.3 KB
scripts/test_plan_says_why.pyscript44.9 KB
scripts/test_provider_inventory.pyscript83.2 KB
scripts/test_provider_names.pyscript126.0 KB
scripts/test_provider_pools.pyscript14.2 KB
scripts/test_public_data.pyscript93.5 KB
scripts/test_quota_exhaustion.pyscript5.6 KB
scripts/test_rate_limit_windows.pyscript6.3 KB
scripts/test_record_says_why.pyscript26.5 KB
scripts/test_reference_provenance.pyscript16.7 KB
scripts/test_refresh_entry.pyscript110.7 KB
scripts/test_released_names.pyscript4.1 KB
scripts/test_reset_priority.pyscript4.8 KB
scripts/test_roll_answers.pyscript44.7 KB
scripts/test_roll_notifications.pyscript10.4 KB
scripts/test_route_judge.pyscript99.8 KB
scripts/test_route_simulation.pyscript7.0 KB
scripts/test_routing_parity.pyscript32.3 KB
scripts/test_routing_review.pyscript25.1 KB
scripts/test_setup_discovery_parity.pyscript9.5 KB
scripts/test_stage_help.pyscript2.0 KB
scripts/test_state_upgrade.pyscript13.8 KB
scripts/test_turn_refusal.pyscript16.7 KB
scripts/test_usage_cache.pyscript4.9 KB
scripts/test_usage_collector.pyscript21.4 KB
scripts/test_usage_core_claude_accounts.pyscript1.8 KB
scripts/test_usage_core_grok_cli_billing.pyscript15.6 KB
scripts/test_usage_core_magpie.pyscript5.4 KB
scripts/test_usage_core_magpie_pool.pyscript10.8 KB
scripts/test_usage_core_safety.pyscript24.4 KB
scripts/test_user_scores.pyscript102.3 KB
scripts/test_venice_usage.pyscript1.3 KB
scripts/test_window_scope.pyscript5.5 KB
scripts/turn_refusal.pyscript10.7 KB
scripts/usage_cache.pyscript4.7 KB
scripts/usage_collector.pyscript7.4 KB
scripts/usage_core.pyscript142.0 KB
scripts/usage_magpie.pyscript10.6 KB
scripts/usage_reading.pyscript17.1 KB
scripts/user_scores.pyscript72.8 KB
scripts/venice_usage.pyscript2.6 KB
scripts/window_scope.pyscript3.2 KB
scripts/worker_context.pyscript8.9 KB
scripts/worker_sessions.pyscript10.9 KB

scripts/test_route_judge.py

View on GitHub
"""Contract tests for route_judge.py: usage snapshots, judge CLI & ordering, Dot probes, and System-One scoring."""
import contextlib
import io
import json
import sys
from datetime import datetime, timedelta, timezone
from pathlib import Path
import tempfile
import time
import unittest
from unittest.mock import patch

import pytest

import account_fleet
import route_judge as router
import decision_ledger
import env_vars
import scorer_client
import subscription_cli
import usage_reading
import task_intake
from router_test_support import isolated_router_state, profile, profile_for, temporary_root


_real_scorer_accounts = scorer_client.scorer_accounts


def setUpModule():
    isolated_router_state(unittest.addModuleCleanup)
    # Scorer contract tests run every family's CLI as if its own account were signed in; ScorerAccountTests
    # covers the per-host choice and the silent skip.
    accounts = patch.object(scorer_client, 'scorer_accounts',
                            side_effect=lambda scorer_type, **_: [scorer_client.SCORER_FAMILIES.get(scorer_type)])
    accounts.start()
    unittest.addModuleCleanup(accounts.stop)


def reference_capabilities():
    return account_fleet.load_capabilities(accounts=account_fleet.load_accounts(account_fleet.reference_fleet_path()))


def live_providers():
    """A fresh, healthy Codex reading, so a plan run on the real clock has one account it can price."""
    live = datetime.now(timezone.utc)
    return {'codex': {'fetched_at': live.isoformat(), 'available': True, 'stale': False,
                      'windows': [{'id': 'usage', 'used_pct': 20, 'resets_at': (live + timedelta(days=3)).isoformat()}]}}


def dot_admitted(result):
    """Whether the plan left dot_text eligible: another account may still be the better pick."""
    row = next((c for c in result['ranking'] if c['id'] == 'dot_text'), None)
    return row is not None and not row['excluded'] and row['score'] is not None


class RoutingTests(unittest.TestCase):
    def setUp(self):
        self.now = datetime(2026, 10, 4, 1, 0, tzinfo=timezone.utc)

    def row(self, used=35, age=30, reset=None, **extra):
        return {'fetched_at': (self.now - timedelta(seconds=age)).isoformat(),
                'available': True, 'stale': False,
                'windows': [{'id': 'usage', 'used_pct': used,
                             'resets_at': reset or (self.now + timedelta(hours=2)).isoformat()}],
                **extra}

    def write_cache(self, directory, providers):
        path = Path(directory) / '.agent-compass' / 'usage-cache.json'
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_text(json.dumps({'providers': providers}), encoding='utf-8')
        return path

    def snapshot(self, providers):
        with tempfile.TemporaryDirectory() as directory:
            return usage_reading.usage_snapshot(self.write_cache(directory, providers), now=self.now)

    def run_plan(self, directory, task, down=None, unavailable=(), records=(), extra=()):
        source = Path(directory) / 'in.json'
        task_with_hours = {'expected_hours': 0.5, **task}
        task_with_hours.setdefault('profile', profile_for(task_with_hours))
        source.write_text(json.dumps({'task': task_with_hours, 'unavailable': list(unavailable)}), encoding='utf-8')
        cache = self.write_cache(directory, live_providers())
        out = io.StringIO()
        with patch.object(sys, 'argv', ['route_judge.py', 'plan', '--dry-run', '--input', str(source),
                                        '--usage-cache', str(cache), *extra]), \
                patch.object(router, 'dot_unreachable', return_value=down) as probe, \
                patch.object(decision_ledger, 'read_ledger', return_value=list(records)), \
                patch.object(decision_ledger, 'LEDGER_WRITES', True), \
                contextlib.redirect_stdout(out):
            router.main()
        return json.loads(out.getvalue()), probe

    def test_a_multi_task_brief_returns_to_the_parent_unless_kept_whole(self):
        profile = dict(GUI_PROFILE, work_kind='architecture_reasoning', needs_gui=False, needs_visual=False,
                       local_only=False, multi_task=True, task_count='3')
        task = {'objective': 'Phase 1: collect references, compare engines, design the loop', 'auto_score': True,
                'profile': profile}
        with tempfile.TemporaryDirectory() as directory:
            split, _ = self.run_plan(directory, task)
            whole, _ = self.run_plan(directory, task, extra=('--keep-whole', 'appointed coordinator of the whole phase'))
            single, _ = self.run_plan(directory, dict(task, profile=dict(profile, multi_task=False, task_count='1')))
        self.assertEqual((split['choice'], split['task_count']), ('split', '3'))
        self.assertIn('--keep-whole', split['next'])
        self.assertNotIn('selected_bundle', split)
        self.assertNotEqual(whole['choice'], 'split')
        self.assertIn('kept_whole', [a['code'] for a in whole['alerts']])
        self.assertNotEqual(single['choice'], 'split')

    def test_multi_task_fields_are_required_and_checked(self):
        self.assertEqual({k: v for k, v in task_intake.validate_profile(dict(GUI_PROFILE, multi_task=True, task_count='>3')).items()
                          if k in ('multi_task', 'task_count')}, {'multi_task': True, 'task_count': '>3'})
        self.assertEqual(task_intake.validate_profile(dict(GUI_PROFILE, multi_task=True, task_count=5))['task_count'], '>3')
        self.assertEqual(
            task_intake.validate_profile({k: v for k, v in dict(GUI_PROFILE, task_count='2').items() if k != 'multi_task'})['multi_task'],
            True,
        )
        for bad in (
            dict(GUI_PROFILE, multi_task='yes', task_count='5'),
            dict(GUI_PROFILE, multi_task='yes', task_count='2'),
            dict(GUI_PROFILE, multi_task=True, task_count='5'),
            dict(GUI_PROFILE, multi_task=False, task_count='2'),
            dict(GUI_PROFILE, multi_task=True, task_count='1'),
            {k: v for k, v in GUI_PROFILE.items() if k != 'task_count'},
        ):
            with self.subTest(bad=bad), self.assertRaises(ValueError):
                task_intake.validate_profile(bad)
        for missing_flag in ('wants_fast', 'local_only', 'needs_visual', 'access'):
            with self.subTest(missing_flag=missing_flag), self.assertRaises(task_intake.InvalidCallerProfile):
                task_intake.validate_caller_profile({k: v for k, v in GUI_PROFILE.items() if k != missing_flag})
        self.assertFalse(hasattr(task_intake, 'multi_task_marks'))
        self.assertFalse(hasattr(task_intake, 'MULTI_TASK_MARK_RE'))
        checked = task_intake.validate_profile(dict(GUI_PROFILE, wants_fast=True, read_only_review=False))
        self.assertEqual((checked['wants_fast'], checked['read_only_review']), (True, False))

    def test_usage_snapshot_discovery_freshness_projection_and_window_math(self):
        with tempfile.TemporaryDirectory() as directory:
            path = self.write_cache(directory, {'codex': self.row()})
            missing = str(Path(directory) / 'missing.json')
            with patch.dict(router.os.environ, {'ROUTER_STATE_DIR': '', 'ROUTER_USAGE_CACHE': ''}), \
                    patch.object(router.Path, 'home', return_value=Path(directory)), \
                    patch.object(env_vars, 'user_env', return_value=None):
                res = usage_reading.usage_snapshot(now=self.now)
            self.assertEqual((res['read_ok'], res['accounts']['codex']['status'],
                              res['accounts']['codex']['windows'][0]['remaining_pct_before_reset']), (True, 'fresh', 65))
            with patch.dict(router.os.environ, {'ROUTER_STATE_DIR': '', 'ROUTER_USAGE_CACHE': ''}), \
                    patch.object(router.Path, 'home', return_value=Path(directory)), \
                    patch.object(env_vars, 'user_env', return_value=missing):
                fallback = usage_reading.usage_snapshot(now=self.now)
            # A named cache file is the only one read: it must not quietly fall back to another copy of the data.
            self.assertEqual((fallback['read_ok'], [a['path'] for a in fallback['attempts']]), (False, [missing]))
            self.assertFalse(usage_reading.usage_snapshot(Path(missing), now=self.now)['read_ok'])

            # Stale & idle cadence
            reset = (self.now + timedelta(days=3)).isoformat()
            stale_path = self.write_cache(directory, {
                'old': self.row(age=901), 'future': self.row(age=-30),
                'reset': self.row(reset='2026-10-04T00:30:00Z'), 'flagged': self.row(stale=True),
                'idle': self.row(age=2000, reset=reset), 'late': self.row(age=2500, reset=reset),
                'quiet': self.row(age=2000, reset=reset),
            })
            doc = json.loads(stale_path.read_text(encoding='utf-8'))
            decided = self.now.timestamp()
            doc['poll'] = {'idle': {'seconds': 1800, 'at': decided - 200},
                           'late': {'seconds': 1800, 'at': decided - 200},
                           'quiet': {'seconds': 1800, 'at': decided - 1000}}
            stale_path.write_text(json.dumps(doc), encoding='utf-8')
            accts = usage_reading.usage_snapshot(stale_path, now=self.now)['accounts']
            for k in ('old', 'future', 'reset', 'flagged', 'late', 'quiet'):
                self.assertEqual(accts[k]['status'], 'stale', k)
            self.assertEqual(accts['idle']['status'], 'fresh')

            # Secret scrubbing & malformed tolerance
            row = self.row(token='secret-canary', summary='secret-canary', reset_block={'credential': 'secret-canary'})
            row['windows'][0]['credential'] = 'secret-canary'
            sec_path = self.write_cache(directory, {'one': row, 'other': self.row()})
            sec_res = usage_reading.usage_snapshot(sec_path, {'one', 'missing'}, self.now)
            self.assertNotIn('secret-canary', json.dumps(sec_res))
            self.assertEqual(sec_res['missing_accounts'], ['missing'])
            sec_path.write_text('{invalid', encoding='utf-8')
            self.assertFalse(usage_reading.usage_snapshot(sec_path, now=self.now)['read_ok'])

        # Window math
        w = usage_reading.window_state({'used_pct': 35, 'resets_at': '2026-10-04T11:00:00+08:00'}, self.now)
        self.assertEqual((w['seconds_until_reset'], w['remaining_pct_before_reset']), (7200, 65))
        pool = usage_reading.window_state({'used_pct': 25, 'accounts': 2, 'resets_at': '2026-10-04T11:00:00+08:00'}, self.now)
        self.assertEqual((pool['used_pct'], pool['remaining_pct_before_reset'], pool['accounts']), (25, 150, 2))
        for odd in (1, 0, True, '2'):
            self.assertEqual(usage_reading.window_state({'used_pct': 0, 'accounts': odd, 'resets_at': '2026-10-04T11:00:00+08:00'},
                                                 self.now)['remaining_pct_before_reset'], 100)
        for over_used in (100.5, 101.0, 110):
            over = usage_reading.window_state({'used_pct': over_used, 'resets_at': '2026-10-04T03:00:00Z'}, self.now)
            self.assertEqual((over['used_pct'], over['observed_remaining_pct'], over['remaining_pct_before_reset']),
                             (100.0, 0.0, 0.0))
        for bad_reset in (None, 'invalid', '2026-10-04T03:00:00'):
            self.assertIsNone(usage_reading.window_state({'used_pct': 80, 'resets_at': bad_reset}, self.now)['remaining_pct_before_reset'])
        for bad_used in (None, '80', -1, float('nan'), float('inf'), True):
            self.assertIsNone(usage_reading.window_state({'used_pct': bad_used, 'resets_at': '2026-10-05T00:00:00Z'}, self.now)['used_pct'])

        # 429 session-limit log detection and rate-limits.json write-back
        with tempfile.TemporaryDirectory() as directory:
            hit_now = datetime(2026, 10, 28, 9, 55, tzinfo=timezone.utc)
            cache = self.write_cache(directory, {
                'claude-lab': {
                    'fetched_at': '2026-10-28T09:38:12Z',
                    'available': True,
                    'stale': True,
                    'windows': [
                        {'id': 'five_hour', 'used_pct': 101.0, 'resets_at': '2026-10-28T13:50:00Z'},
                        {'id': 'seven_day', 'used_pct': 74.0, 'resets_at': '2026-10-28T15:00:00Z'},
                    ],
                }
            })
            log_path = Path(directory) / '.claude-lab' / 'projects' / 'E--PROJECTS-route-manager' / 'sess1.jsonl'
            log_path.parent.mkdir(parents=True, exist_ok=True)
            log_path.write_text(json.dumps({
                'type': 'assistant',
                'timestamp': '2026-10-28T09:52:07.000Z',
                'isApiErrorMessage': True,
                'error': 'rate_limit',
                'message': {'content': [{'type': 'text', 'text': "API Error: Request rejected (429) · Claude Code: You've hit your session limit · resets 1:50pm (UTC)"}]},
            }) + '\n', encoding='utf-8')
            snap_429 = usage_reading.usage_snapshot(cache, now=hit_now)
            c2 = snap_429['accounts']['claude-lab']
            self.assertEqual(c2.get('rate_limited_until'), '2026-10-28T13:50:00+00:00')
            w5 = next(w for w in c2['windows'] if w['id'] == 'five_hour')
            self.assertEqual((w5['used_pct'], w5['observed_remaining_pct'], w5['remaining_pct_before_reset'], w5.get('rate_limited')),
                             (100.0, 0.0, 0.0, True))
            persisted = json.loads((Path(directory) / '.agent-compass' / 'rate-limits.json').read_text(encoding='utf-8'))
            self.assertEqual(persisted['claude-lab']['resets_at'], '2026-10-28T13:50:00+00:00')

    def test_subscription_env_drops_api_keys_and_points_at_the_account_home(self):
        with patch.dict(router.os.environ, {'OPENAI_API_KEY': 'secret', 'ROUTER_CUSTOM': 'keep'}), \
                patch.object(router.provider_inventory, 'current', return_value={'accounts': {
                    'claude-lab': {'family': 'claude', 'home': str(Path.home() / '.claude-lab')}}}):
            env = subscription_cli.subscription_env('claude-lab')
            self.assertNotIn('OPENAI_API_KEY', env)
            self.assertEqual(env['ROUTER_CUSTOM'], 'keep')
            self.assertTrue(env['CLAUDE_CONFIG_DIR'].endswith('.claude-lab'))

    def test_capacity_alerts_ledger_and_constraint_recording(self):
        reset = (self.now + timedelta(days=3)).isoformat()
        proxy_row = self.row(used=0, reset=reset)
        proxy_row['billing'] = 'metered'
        usage = self.snapshot({'codex': self.row(used=90, reset=reset), 'claude': self.row(used=5, reset=reset),
                               'team-proxy': proxy_row, 'old': self.row(age=15 * 86400)})
        self.assertTrue(usage['accounts']['old']['flag_conflict'])
        norm_reset = usage['accounts']['codex']['windows'][0]['resets_at']
        earlier = [{'usage': {'codex': {'fetched_at': (self.now - timedelta(hours=h)).isoformat(),
                                        'windows': [{'id': 'usage', 'used_pct': used, 'resets_at': norm_reset}]}}}
                   for h, used in ((5, 40), (2, 70))]
        alerts = decision_ledger.capacity_alerts(usage, ['codex'], earlier, self.now)
        self.assertEqual({a['code'] for a in alerts}, {'low_remaining', 'exhaustion_before_reset', 'idle_alternative'})
        self.assertEqual([a['account'] for a in alerts if a['code'] == 'idle_alternative'], ['claude'])

        # Finalize & record_constraint
        now = datetime.now(timezone.utc)
        pro = {'provider': 'chatgpt-web', 'account': 'fixture-advisor', 'model': 'gpt-6-pro', 'effort': 'max'}
        accounts = account_fleet.load_accounts(account_fleet.reference_fleet_path())
        accounts['accounts']['fixture-advisor'] = {'cadence': {'window_seconds': 6 * 3600, 'limit': 5}}
        history = [{'at': (now - timedelta(hours=h)).isoformat(), 'stage': 'record', 'choice': 'constraint', 'bundle': pro} for h in (1, 2, 3, 4, 5)]
        res = {'stage': 'record', 'choice': 'constraint', 'source': 'constraint_record', 'constraint': 'user_override',
               'basis': 'User: ask Pro', 'selected_bundle': pro, 'usage': {'accounts': {}}}
        with tempfile.TemporaryDirectory() as directory, \
                patch.object(decision_ledger, 'ledger_path', return_value=Path(directory) / 'ledger.jsonl'), \
                patch.object(account_fleet, 'load_accounts', return_value=accounts):
            with patch.object(decision_ledger, 'LEDGER_WRITES', False):     # the alerts only: no reservation for a fixture row
                out = decision_ledger.finalize(res, time.monotonic(),
                                               {'objective': 'Advise', 'work_kind': 'architecture_reasoning'}, history)
            self.assertEqual({a['code'] for a in out['alerts']}, {'advisor_cadence', 'telemetry_missing'})
            args = router.argparse.Namespace(basis='User: Sol xhigh', provider='codex', account='codex',
                                             model='gpt-6.1-sol', effort='xhigh', input=None, usage_cache=None,
                                             accounts=None, constraint='user_override', specialty=None)
            with patch.object(usage_reading, 'usage_snapshot', return_value={'read_ok': True, 'accounts': {}}):
                rec = router.record_constraint(args)
            self.assertEqual(rec['source'], 'constraint_record')
            self.assertNotIn('trigger', rec)
            # A record that authorizes a Workflow says so in the ledger row the spawn gate reads.
            args.trigger = 'workflow'
            with patch.object(usage_reading, 'usage_snapshot', return_value={'read_ok': True, 'accounts': {}}):
                rec = router.record_constraint(args)
            self.assertEqual(rec['trigger'], 'workflow')
            decision_ledger.finalize(rec, time.monotonic(), {'parent_agent_id': 'session-1'}, [])
            row = json.loads((Path(directory) / 'ledger.jsonl').read_text(encoding='utf-8').splitlines()[-1])
            self.assertEqual((row['stage'], row['trigger'], row['constraint']), ('record', 'workflow', 'user_override'))
            del args.trigger
            args.account = 'claude-lab'
            args.provider = 'claude-lab'
            args.model = 'claude-opus-5-5'
            # The model-management checkbox governs automatic routing only: the user's naming launches it with a
            # warning, a project rule does not (router-internals.md, Pins).
            with patch.object(usage_reading, 'usage_snapshot', return_value={'read_ok': True, 'accounts': {}}):
                named = router.record_constraint(args)
            self.assertTrue(any('unchecked in model management' in w for w in named.get('warnings') or ()), named)
            args.constraint = 'hard_constraint'
            with patch.object(usage_reading, 'usage_snapshot', return_value={'read_ok': True, 'accounts': {}}),                     self.assertRaisesRegex(ValueError, 'unchecked in model management'):
                router.record_constraint(args)

    def test_ledger_rows_name_the_writing_session_not_its_parent(self):
        """The spawn gate binds Workflow records and schedule plans to `session_ids`; parent_id stays the parent link."""
        import roll
        import worker_context
        session_vars = [*worker_context.HOST_SESSION_ID_VARS, 'PASEO_AGENT_ID']
        cases = (
            # A roll-launched CLI worker: roll hands it the parent link, its own host sets its session.
            ({'AGENTCOMPASS_WORKER': '1', 'ROUTER_PARENT_AGENT_ID': 'P', 'CLAUDE_CODE_SESSION_ID': 'W'}, ['W']),
            # A main Claude session under Paseo: its host session and its own Paseo agent.
            ({'CLAUDE_CODE_SESSION_ID': 'S1', 'PASEO_AGENT_ID': 'A1'}, ['S1', 'A1']),
            ({'ROUTER_PARENT_AGENT_ID': 'P'}, None),     # only a parent link: no writer identity at all
        )
        for env, expected in cases:
            with self.subTest(env=env), tempfile.TemporaryDirectory() as directory, \
                    patch.dict('os.environ', {var: '' for var in session_vars}), \
                    patch.object(decision_ledger, 'ledger_path', return_value=Path(directory) / 'ledger.jsonl'):
                import os
                for var in session_vars:
                    os.environ.pop(var, None)
                os.environ.update(env)
                self.assertEqual(worker_context.own_session_ids(), expected or [])
                for stage in ('plan', 'record'):
                    decision_ledger.finalize({'stage': stage, 'choice': 'x', 'source': 's', 'selected_bundle': {}},
                                             time.monotonic(), {'parent_agent_id': 'P'}, [])
                rows = [json.loads(line) for line in
                        (Path(directory) / 'ledger.jsonl').read_text(encoding='utf-8').splitlines()]
                self.assertEqual([row['stage'] for row in rows], ['plan', 'record'])
                for row in rows:
                    self.assertEqual(row.get('session_ids'), expected)
        # roll never passes the parent's own session identity on to a CLI worker.
        for var in worker_context.OWN_SESSION_ID_VARS + ('PASEO_AGENT_ID',):
            self.assertTrue(var.startswith(roll.HOST_SESSION_ENV), var)

    def test_constraint_record_cannot_bypass_local_account_pause_with_an_unknown_model(self):
        args = router.argparse.Namespace(basis='Synthetic explicit model request', provider='codex', account='codex',
            model='new-model', effort='high', input=None, usage_cache=None,
            accounts=None, constraint='user_override', specialty=None)
        snapshot = {'accounts': {'codex': {'family': 'codex', 'status': 'paused',
                    'reason': 'unhealthy_in_cc_switch', 'plan': 'pro'}}, 'launchers': {}}
        with patch.object(router.provider_inventory, 'current', return_value=snapshot), \
                patch.object(usage_reading, 'usage_snapshot', return_value={'read_ok': True, 'accounts': {}}):
            with self.assertRaisesRegex(ValueError, 'unhealthy_in_cc_switch'):
                router.record_constraint(args)

    def test_dot_probe_and_caller_local_dependency_admission(self):
        class Ready:
            status = 200
            def __enter__(self): return self
            def __exit__(self, *exc): return False

        with tempfile.TemporaryDirectory() as directory:
            config = Path(directory) / '.dot2api' / 'client.json'
            config.parent.mkdir()
            config.write_text(json.dumps({'base_url': 'http://127.0.0.1:8788/', 'api_key': 'secret-key'}), encoding='utf-8')
            with patch.object(router.Path, 'home', return_value=Path(directory)), \
                    patch.dict(router.os.environ, {'DOT2API_BASE_URL': ''}), \
                    patch.object(router.urllib.request, 'urlopen', return_value=Ready()):
                self.assertIsNone(router.dot_unreachable())

            cases = [
                ('remote', {}, None, True, True, 'dot_handoff'),
                ('local runtime', {'local_only': True, 'local_only_reason': 'runs local pytest'}, None, False, False, None),
                ('missing reason respected', {'local_only': True}, None, False, False, None),
                ('edit-only reason respected', {'local_only': True, 'local_only_reason': 'in-place edits of workspace files'},
                 None, False, False, None),
                ('explicit false', {'local_only': False}, None, True, True, 'dot_handoff'),
                ('credentials', {'local_only': True, 'local_only_reason': 'Needs local ssh-agent VPS credentials'},
                 None, False, False, None),
                ('unreachable', {}, 'URLError on /readyz', False, True, 'dot_unreachable'),
            ]
            for label, needs, down, dot, probed, alert in cases:
                with self.subTest(case=label):
                    result, probe = self.run_plan(directory, {'objective': 'Fix single-file parser bug',
                        'work_kind': 'code_implementation', 'difficulty': 'hard', 'needs': needs}, down)
                    self.assertEqual(dot_admitted(result), dot)
                    self.assertNotEqual(result['choice'], 'none')
                    self.assertEqual(probe.called, probed)
                    codes = {a['code'] for a in result['alerts']}
                    if alert and (alert != 'dot_handoff' or result['choice'] == 'dot_text'):
                        self.assertIn(alert, codes)
                    self.assertNotIn('local_only_ignored_without_reason', codes)
            ssh, _ = self.run_plan(directory, {'objective': 'Read-only checklist over SSH and the repository',
                'work_kind': 'live_loop', 'difficulty': 'hard', 'needs': {'local_only': True}})
            self.assertNotIn(ssh['choice'], ('dot_text', 'none'))
            unknown, _ = self.run_plan(directory, {'objective': 'Compute', 'work_kind': 'research_compute', 'difficulty': 'hard'})
            self.assertIn('unknown_work_kind', {a['code'] for a in unknown['alerts']})

    def test_recent_dot_reroll_requires_local_evidence_or_failed_health(self):
        capabilities = reference_capabilities()
        task = {'task_id': 't-dot-1', 'objective': 'Fix single-file parser bug', 'work_kind': 'code_implementation', 'difficulty': 'hard'}
        recent = {'at': datetime.now(timezone.utc).isoformat(), 'stage': 'plan',
                  'choice': 'dot_text', 'task_id': 't-dot-1', 'request_hash': decision_ledger.task_request_hash(task)}
        for local_only, down, enabled, dot, blocked in (
            (False, None, True, True, True),
            (True, None, True, False, False),
            (False, 'endpoint down', True, False, False),
            (False, None, False, False, False),
        ):
            with self.subTest(local_only=local_only, down=down, enabled=enabled), \
                    tempfile.TemporaryDirectory() as directory, \
                    patch.object(account_fleet, 'load_capabilities', return_value={**capabilities,
                        'remote_reroute_guard': {'enabled': enabled, 'bundle_id': 'dot_text', 'window_seconds': 600}}):
                result, _ = self.run_plan(directory, dict(task, needs={'local_only': local_only}),
                                           down, unavailable=['dot_text'], records=[recent])
                self.assertEqual(dot_admitted(result), dot)
                self.assertNotEqual(result['choice'], 'none')
                self.assertEqual('dot_reroll_blocked' in {a['code'] for a in result['alerts']}, blocked)
        # m1: sharing an objective prefix (>180 chars) without matching task_id/chain_id/request_hash never locks
        prefix = 'Shared preamble ' * 15
        rec_prefix = {'at': datetime.now(timezone.utc).isoformat(), 'stage': 'plan', 'choice': 'dot_text',
                      'objective': prefix + 'task one',
                      'request_hash': decision_ledger.task_request_hash({'objective': prefix + 'task one'})}
        self.assertIsNone(router.recent_route_lock(
            {'objective': prefix + 'task two'}, [rec_prefix],
            datetime.now(timezone.utc), 'dot_text', 600))
        self.assertIsNotNone(router.recent_route_lock(
            {'objective': prefix + 'task one'}, [rec_prefix],
            datetime.now(timezone.utc), 'dot_text', 600))

    @pytest.mark.slow
    def test_timeout_stops_owned_process(self):
        with tempfile.TemporaryDirectory() as directory:
            with self.assertRaises(RuntimeError):
                subscription_cli.run_process([sys.executable, '-c', 'import time; time.sleep(30)'], directory, timeout=0.1)


def as_agy_stream(stdout):
    """agy answers on its stream-json channel: a scripted single result envelope arrives as a `result` event."""
    try:
        envelope = json.loads(stdout)
    except (TypeError, ValueError):
        return stdout
    if not isinstance(envelope, dict) or 'event' in envelope:
        return stdout
    return json.dumps({'event': 'init'}) + chr(10) + json.dumps({'event': 'result', 'result': envelope}) + chr(10)


GUI_PROFILE = {
    'work_kind': 'gui_control',
    'primary_level': 1,
    'secondary_kind': 'none',
    'secondary_level': 0,
    'local_only': True,
    'needs_visual': True,
    'wants_fast': False,
    'needs_gui': True,
    'read_only_review': False,
    'access': 'write',
    'task_count': '1',
    'multi_task': False,
    'axis_demand': {'creativity': 0, 'coding': 0, 'aesthetics': 0, 'tool_operation': 3},
}


class ReadinessGapTests(unittest.TestCase):
    """A router with nothing to route to, or no quota data at all, must say so instead of returning a confident pick."""

    @staticmethod
    def accounts(*statuses):
        return {'_discovery': {'accounts': {f'a{i}': {'status': s, 'family': 'claude'} for i, s in enumerate(statuses)}}}

    def test_readiness_gap_severity_and_effect_on_choice(self):
        for statuses, read_ok, code, fatal in (
            (('missing_credentials', 'missing_client'), True, 'no_ready_account', True),
            (('paused',), True, 'no_ready_account', True),
            (('ready',), False, 'no_usage_data', False),
            (('ready',), True, None, False),
            (None, True, None, False),
        ):
            with self.subTest(statuses=statuses, read_ok=read_ok):
                accounts = self.accounts(*statuses) if statuses is not None else {}
                usage = {'read_ok': read_ok}
                gap = router.readiness_gap(accounts, usage)
                planned = {'choice': 'sonnet55_high', 'selected_bundle': {'account': 'a0'}, 'decision': {}}
                router.apply_readiness_gap(planned, accounts, usage)
                if code is None:
                    self.assertIsNone(gap)
                else:
                    self.assertEqual((gap['code'], gap['fatal']), (code, fatal))
                if fatal:
                    self.assertIn(' setup` to detect the CLIs', gap['message'])
                    self.assertNotIn('onboard.py', gap['message'])
                    self.assertEqual((planned['choice'], planned['error']), ('none', code))
                    self.assertNotIn('selected_bundle', planned)
                    self.assertEqual(planned['alerts'][0]['severity'], 'warn')
                    self.assertIn(' setup`', planned['decision']['reason'])
                else:
                    self.assertEqual(planned['choice'], 'sonnet55_high')
                    self.assertEqual(planned['selected_bundle'], {'account': 'a0'})

    def test_plan_with_ready_account_but_no_cache_warns(self):
        inventory = {'accounts': {'antigravity': {'family': 'antigravity', 'status': 'ready'}}, 'launchers': {}}
        with tempfile.TemporaryDirectory() as directory:
            source = Path(directory) / 'in.json'
            source.write_text(json.dumps({'task': {'objective': 'x', 'expected_hours': 0.5, 'work_kind': 'code_implementation',
                                                   'difficulty': 'routine', 'profile': profile()}}), encoding='utf-8')
            out = io.StringIO()
            missing = str(Path(directory) / '.usage-float' / 'usage-cache.json')
            with patch.object(sys, 'argv', ['route_judge.py', 'plan', '--dry-run', '--no-score', '--input', str(source),
                                            '--usage-cache', missing]), \
                    patch.object(router.provider_inventory, 'current', return_value=inventory), \
                    patch.object(router, 'dot_unreachable', return_value='down'), \
                    patch.object(decision_ledger, 'read_ledger', return_value=[]), contextlib.redirect_stdout(out):
                router.main()
        parsed_out = json.loads(out.getvalue())
        alerts = {a['code']: a['severity'] for a in parsed_out['alerts']}
        self.assertEqual(alerts.get('no_usage_data'), 'warn')


class CliPrefixTests(unittest.TestCase):
    def test_grok_installed_outside_path_resolves_to_its_home_binary(self):
        with tempfile.TemporaryDirectory() as tmp:
            installed = Path(tmp) / '.grok' / 'bin' / ('grok.exe' if router.os.name == 'nt' else 'grok')
            with patch.object(subscription_cli.shutil, 'which', return_value=None), patch.object(router.Path, 'home', return_value=Path(tmp)):
                with self.assertRaisesRegex(RuntimeError, 'not installed'):
                    subscription_cli.cli_prefix('grok')
                installed.parent.mkdir(parents=True)
                installed.write_bytes(b'')
                self.assertEqual(subscription_cli.cli_prefix('grok'), [str(installed)])
                with self.assertRaisesRegex(RuntimeError, 'not installed'):
                    subscription_cli.cli_prefix('codex')
            with patch.object(subscription_cli.shutil, 'which', return_value='C:/tools/grok.exe'):
                self.assertEqual(subscription_cli.cli_prefix('grok'), [str(Path('C:/tools/grok.exe'))])


class ScoreCacheTests(unittest.TestCase):
    """Classification cache: same task and scorer identity skips the scorer call; anything else misses."""

    def setUp(self):
        self.capabilities = reference_capabilities()
        self.dir = tempfile.TemporaryDirectory()
        self.addCleanup(self.dir.cleanup)
        patcher = patch.dict(router.os.environ, {'ROUTER_SCORE_CACHE': str(Path(self.dir.name) / 'score-cache.json')})
        patcher.start()
        self.addCleanup(patcher.stop)
        self.calls = 0

    def score(self, objective='Fix the parser bug', config=None, stdout=None):
        def run(command, cwd, prompt=None, timeout=120, account=None):
            self.calls += 1
            answer = stdout or json.dumps({'status': 'SUCCESS', 'structured_output': GUI_PROFILE})
            return 0, (as_agy_stream(answer) if command[0] == 'agy.exe' else answer), ''
        cfg = config if config is not None else {'scorer_type': 'agy_flash_low', 'fallback_to_agy': False}
        with patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), patch.object(subscription_cli, 'run_process', side_effect=run):
            return scorer_client.score_task({'objective': objective}, self.capabilities, scorer_config=cfg)

    def test_repeat_is_served_from_cache_and_changed_task_or_scorer_is_not(self):
        first, meta = self.score()
        again, cached = self.score()
        self.assertEqual((self.calls, again, cached.get('cached'), meta.get('cached')), (1, first, True, None))
        self.score('Fix the lexer bug')
        self.assertEqual(self.calls, 2)
        self.score(config={'scorer_type': 'jev', 'jev_api_key': 'k1', 'fallback_to_agy': True})
        self.assertEqual(self.calls, 3)  # different scorer identity (the jev call fails, cascade answers)

    def test_corrupt_stale_and_fallback_results_never_serve_from_cache(self):
        self.score()
        path = scorer_client.score_cache_path()
        path.write_text('{not json', encoding='utf-8')
        self.score()
        self.assertEqual(self.calls, 2)
        data = json.loads(path.read_text(encoding='utf-8'))
        for entry in data['entries'].values():
            entry['saved_at'] -= scorer_client.SCORE_CACHE_TTL_SECONDS + 1
        path.write_text(json.dumps(data), encoding='utf-8')
        self.score()
        self.assertEqual(self.calls, 3)
        path.unlink()
        prof, meta = self.score('Other task', stdout='garbage')
        self.assertIsNone(prof)
        self.assertEqual(meta['source'], 'scorer_unavailable')
        self.score('Other task', stdout='garbage')
        self.assertEqual(self.calls, 5)

    def test_prefetch_classifies_tasks_concurrently_and_every_result_reaches_the_cache(self):
        import threading
        running, peak, lock = [0], [0], threading.Lock()

        def run(command, cwd, prompt=None, timeout=120, account=None):
            with lock:
                running[0] += 1
                peak[0] = max(peak[0], running[0])
            time.sleep(0.05)
            with lock:
                running[0] -= 1
                self.calls += 1
            return 0, as_agy_stream(json.dumps({'status': 'SUCCESS', 'structured_output': GUI_PROFILE})), ''
        tasks = [{'objective': f'Fix bug {n}'} for n in range(4)]
        with patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), patch.object(subscription_cli, 'run_process', side_effect=run), \
                patch.object(scorer_client, 'load_scorer_config', return_value={'scorer_type': 'agy_flash_low', 'fallback_to_agy': False}), \
                patch.object(account_fleet, 'load_capabilities', return_value=self.capabilities):
            scorer_client.prefetch_scores(tasks)
        self.assertEqual(self.calls, 4)
        self.assertGreater(peak[0], 1)
        for n in range(4):
            self.assertTrue(self.score(f'Fix bug {n}')[1].get('cached'))
        self.assertEqual(self.calls, 4)

    def test_cache_holds_only_the_classification_never_routing_state_or_keys(self):
        self.score(config={'scorer_type': 'jev', 'jev_api_key': 'secret-key-123', 'fallback_to_agy': True})
        self.score()
        raw = scorer_client.score_cache_path().read_text(encoding='utf-8')
        self.assertNotIn('secret-key-123', raw)
        for entry in json.loads(raw)['entries'].values():
            self.assertEqual(set(entry), {'saved_at', 'profile', 'scorer'})

    def test_cache_is_bounded(self):
        for index in range(scorer_client.SCORE_CACHE_MAX + 5):
            scorer_client.score_cache_put(f'k{index}', GUI_PROFILE, {'source': 'system_one'})
        self.assertEqual(len(scorer_client._score_cache_read()), scorer_client.SCORE_CACHE_MAX)


class FullTaskTextTests(unittest.TestCase):
    """The classifier gets the whole task text on every transport; a text no transport can carry stops instead of being cut."""

    def setUp(self):
        self.capabilities = reference_capabilities()
        patcher = patch.dict(router.os.environ, {'ROUTER_SCORE_CACHE': 'off', 'ROUTER_DISABLE_LIVE_SCORER': ''})
        patcher.start()
        self.addCleanup(patcher.stop)
        crlf = chr(13) + chr(10)
        filler = ('普通背景说明,请忽略。' + crlf) * 2000  # ~24k characters, far past the agy command line
        self.text = filler + '中间的硬性验收条件:必须在周五前完成。' + crlf + filler
        self.config = {'scorer_type': 'agy_flash_low', 'fallback_to_agy': True}

    def test_evidence_is_never_cut_and_the_middle_is_part_of_the_prompt(self):
        prompt = scorer_client.scorer_prompt({'objective': self.text, 'context': '上下文' * 5000})
        self.assertIn(json.dumps(self.text, ensure_ascii=False), prompt)
        self.assertIn('中间的硬性验收条件', prompt)
        self.assertNotEqual(scorer_client.score_cache_key({'objective': self.text}, self.capabilities, ['agy_flash_low'], {}),
                            scorer_client.score_cache_key({'objective': self.text.replace('周五', '周四')},
                                                   self.capabilities, ['agy_flash_low'], {}))

    def test_agy_sends_every_task_on_stdin_and_nothing_long_on_the_command_line(self):
        runs = []

        def fake_run(command, cwd, prompt=None, timeout=120, account=None):
            runs.append((command, prompt))
            result = {'conversation_id': 'c', 'status': 'SUCCESS', 'structured_output': GUI_PROFILE}
            return 0, json.dumps({'event': 'init'}) + chr(10) + json.dumps({'event': 'result', 'result': result}) + chr(10), ''
        with patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), patch.object(subscription_cli, 'run_process', side_effect=fake_run):
            scorer_client.score_task({'objective': self.text}, self.capabilities, scorer_config=self.config)
            scorer_client.score_task({'objective': 'short task'}, self.capabilities, scorer_config=self.config)
        for (command, stdin), expected in zip(runs, (self.text, 'short task')):
            self.assertEqual(command[-2:], ['--print', ''])
            self.assertIn('stream-json', command)
            message = json.loads(stdin)
            self.assertEqual(message['event'], 'user')
            self.assertIn(json.dumps(expected, ensure_ascii=False), message['message']['content'])
            self.assertIn('calibration_cases', message['message']['content'])
            self.assertLess(max(len(part) for part in command), 8000)  # only the compact schema rides the command line

    def test_a_failed_stdin_channel_gives_scorer_unavailable_for_any_task_length(self):
        with patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), \
                patch.object(subscription_cli, 'run_process', return_value=(2, '', 'unsupported')):
            for objective in (self.text, 'short task'):
                prof, meta = scorer_client.score_task({'objective': objective}, self.capabilities, scorer_config=self.config)
                self.assertIsNone(prof)
                self.assertEqual(meta['source'], 'scorer_unavailable')

    def test_api_scorer_refusing_the_size_falls_back_to_agy_stdin_not_to_heuristic(self):
        ok = {'conversation_id': 'c', 'status': 'SUCCESS', 'structured_output': GUI_PROFILE}
        error = router.urllib.error.HTTPError('https://api.typesafe.ai', 413, 'too large', {}, None)
        with patch.object(router.urllib.request, 'urlopen', side_effect=error), patch.object(env_vars, 'user_env', return_value=None),                 patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']),                 patch.object(subscription_cli, 'run_process', return_value=(0, json.dumps({'event': 'result', 'result': ok}), '')):
            _, meta = scorer_client.score_task({'objective': self.text}, self.capabilities,
                                        scorer_config={'scorer_type': 'jev', 'jev_api_key': 'k', 'fallback_to_agy': True})
            self.assertEqual((meta['source'], meta['attempts'][0]['error_type']), ('system_one', 'ScorerInputError'))
            with self.assertRaises(scorer_client.ScorerInputError):
                scorer_client.score_task({'objective': self.text}, self.capabilities,
                                  scorer_config={'scorer_type': 'jev', 'jev_api_key': 'k', 'fallback_to_agy': False})


class SystemOneScorerTests(unittest.TestCase):
    """Contract tests for System-One scoring via Gemini 3.8 Flash low and heuristic fallback."""

    def setUp(self):
        self.capabilities = reference_capabilities()
        # Tests rescore one objective with different fake answers; a cache hit would replay the first.
        patcher = patch.dict(router.os.environ, {'ROUTER_SCORE_CACHE': 'off'})
        patcher.start()
        self.addCleanup(patcher.stop)

    def envelope(self, structured=None, response='', status='SUCCESS'):
        return json.dumps({'conversation_id': 'c', 'status': status, 'response': response,
                           **({'structured_output': structured} if structured is not None else {})})

    def fake_agy(self, stdout, code=0):
        calls = []
        def run(command, cwd, prompt=None, timeout=120, account=None):
            calls.append({'command': command, 'cwd': cwd, 'timeout': timeout, 'stdin': prompt})
            return code, as_agy_stream(stdout), ''
        return calls, run

    def score(self, task, stdout=None, code=0, use_model=True):
        calls, run = self.fake_agy(stdout if stdout is not None else self.envelope(GUI_PROFILE), code)
        with patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), patch.object(subscription_cli, 'run_process', side_effect=run):
            profile, meta = scorer_client.score_task(
                task, self.capabilities, use_model=use_model,
                scorer_config={'scorer_type': 'agy_flash_low', 'fallback_to_agy': False},
            )
        return profile, meta, calls

    def run_cli(self, directory, stage, task, *flags, stdout=None, dot=None):
        source = Path(directory) / 'in.json'
        payload_task = {'expected_hours': 0.5, **task} if stage == 'plan' and 'expected_hours' not in task and '--hours' not in flags else task
        source.write_text(json.dumps({'task': payload_task}), encoding='utf-8')
        cache = Path(directory) / 'usage-cache.json'
        cache.write_text(json.dumps({'providers': live_providers()}), encoding='utf-8')
        calls, run = self.fake_agy(stdout if stdout is not None else self.envelope(GUI_PROFILE))
        out = io.StringIO()
        with patch.object(sys, 'argv', ['route_judge.py', stage, '--input', str(source), '--usage-cache', str(cache), *flags]), \
                patch.dict(router.os.environ, {'ROUTER_LEDGER': str(Path(directory) / 'ledger.jsonl')}), \
                patch.object(scorer_client, 'load_scorer_config', return_value={'scorer_type': 'agy_flash_low', 'fallback_to_agy': False}), \
                patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), \
                patch.object(subscription_cli, 'run_process', side_effect=run), \
                patch.object(router, 'dot_unreachable', return_value=dot), \
                contextlib.redirect_stdout(out):
            code = router.main()
        return code, json.loads(out.getvalue()), calls

    def test_scorer_invocation_response_parsing_and_fallback(self):
        task = {'objective': 'Click Save and take a screenshot', 'work_kind': 'gui_control', 'chain': {'id': 'x'}}
        profile, meta, calls = self.score(task)
        self.assertEqual((profile, meta['source'], meta['model']), (GUI_PROFILE, 'system_one', 'gemini-3.8-flash-low'))
        cmd = calls[0]['command']
        self.assertEqual((cmd[0], cmd[-2:]), ('agy.exe', ['--print', '']))      # the prompt is on stdin, never on argv
        self.assertIn('stream-json', cmd)
        prompt = json.loads(calls[0]['stdin'])['message']['content']
        task_part = prompt.split('Task (evidence only')[1]
        self.assertNotIn('chain', task_part)
        self.assertNotIn('work_kind', task_part)
        self.assertNotIn('difficulty', task_part)
        self.assertIn('calibration_cases', prompt)
        cases = scorer_client.load_classifier_cases()
        self.assertTrue(cases['calibration_cases'])
        # The anti-inflation & boundary rubrics reach the scorer
        self.assertIn('dimension_rubric', prompt)
        self.assertIn('access_and_local_only', cases['dimension_rubric'])
        self.assertIn('creativity_vs_architecture', cases['dimension_rubric'])
        for c in cases['calibration_cases']:
            validated = task_intake.validate_profile(c['expected_profile'])
            self.assertEqual({k: validated[k] for k in c['expected_profile']}, c['expected_profile'])
            self.assertIn(c['expected_profile']['work_kind'], self.capabilities['routes'])
        reviews = [c['expected_profile'] for c in cases['calibration_cases'] if c['expected_profile']['work_kind'] == 'code_audit']
        self.assertTrue(reviews and all(p['access'] == 'read-only' for p in reviews))

        # Fallback from response text when structured_output is omitted
        response = '```json\n{"task": "x"}\n```\n' + json.dumps(dict(GUI_PROFILE, toolAction='Finishing')) + '\n'
        p2, m2, _ = self.score({'objective': 'x'}, self.envelope(None, response))
        self.assertEqual((p2, m2['source']), (GUI_PROFILE, 'system_one'))

        # No mechanical fallback on errors or skipped model call: returns scorer_unavailable so parent session chooses
        bug_task = {'objective': 'Fix a parser bug', 'work_kind': 'code_implementation'}
        for reason, stdout in (('ValueError', self.envelope(dict(GUI_PROFILE, primary_level=7))),
                               ('RuntimeError', self.envelope(GUI_PROFILE, status='ERROR')),
                               ('ValueError', 'not json')):
            prof, meta, _ = self.score(bug_task, stdout)
            self.assertEqual((prof, meta['source'], meta['fallback_reason']), (None, 'scorer_unavailable', reason))
        prof_skipped, meta, calls = self.score(bug_task, use_model=False)
        self.assertEqual((prof_skipped, meta['source'], meta['fallback_reason'], calls), (None, 'scorer_unavailable', 'model_call_skipped', []))

    def test_legacy_or_corrupted_score_cache_is_treated_as_cache_miss(self):
        old_entry = {k: v for k, v in GUI_PROFILE.items() if k not in ('wants_fast', 'access')}
        contradictory_entry = dict(GUI_PROFILE, work_kind='code_audit', access='write')
        config = {'scorer_type': 'agy_flash_low', 'fallback_to_agy': False}
        _, order = scorer_client.scorer_attempt_order(config)
        for bad_cached in (old_entry, contradictory_entry):
            with self.subTest(bad_cached=bad_cached), tempfile.TemporaryDirectory() as directory:
                cache_file = Path(directory) / 'score-cache.json'
                task = {'objective': 'Click Save and take a screenshot', 'expected_hours': 0.5}
                with patch.dict(router.os.environ, {'ROUTER_SCORE_CACHE': str(cache_file)}):
                    key = scorer_client.score_cache_key(task, self.capabilities, order, config)
                    cache_file.write_text(
                        json.dumps({'entries': {key: {'saved_at': time.time(), 'profile': bad_cached,
                                                      'scorer': {'source': 'system_one', 'model': 'old'}}}}),
                        encoding='utf-8',
                    )
                    profile, meta, calls = self.score(task)
                self.assertEqual(len(calls), 1)
                self.assertEqual(profile, GUI_PROFILE)
                self.assertFalse(meta.get('cached', False))

    def test_score_and_plan_with_score_cli_contracts(self):
        with tempfile.TemporaryDirectory() as directory:
            code, res, calls = self.run_cli(directory, 'score', {'objective': 'Click Save and take a screenshot'})
            self.assertEqual((code, len(calls), res['stage'], res['task_profile']), (0, 1, 'score', GUI_PROFILE))
            self.assertFalse((Path(directory) / 'ledger.jsonl').exists())

            gui_task = {'objective': 'Click Save in local app', 'work_kind': 'gui_control',
                        'needs': {'local_only': True, 'local_only_reason': 'operates local screen'}}
            code, planned, _ = self.run_cli(directory, 'plan', gui_task, '--score')
            self.assertEqual((code, planned['scorer']['source'], planned['decision']['task']['difficulty']), (0, 'system_one', 'routine'))

            # Default plan without --score and without caller work_kind/difficulty invokes classifier automatically
            bare_gui = {'objective': 'Click Save in local app',
                        'needs': {'local_only': True, 'local_only_reason': 'operates local screen'}}
            code, auto_planned, auto_calls = self.run_cli(
                directory, 'plan', bare_gui, stdout=self.envelope(dict(GUI_PROFILE, work_kind='gui_control')))
            self.assertEqual((code, len(auto_calls), auto_planned['scorer']['source'],
                              auto_planned['decision']['task']['kind'], auto_planned['decision']['task']['difficulty']),
                             (0, 1, 'system_one', 'gui_control', 'routine'))

            # Inflated caller difficulty / manual work_kind is overridden by the System-One classifier
            _, inflated, _ = self.run_cli(
                directory, 'plan', {'objective': 'Click Save', 'work_kind': 'architecture_reasoning', 'difficulty': 'coupled'})
            alert_codes = [a['code'] for a in inflated['alerts']]
            self.assertIn('difficulty_above_profile', alert_codes)
            self.assertIn('caller_classification_ignored', alert_codes)
            self.assertEqual((inflated['decision']['task']['kind'], inflated['decision']['task']['difficulty']),
                             ('gui_control', 'routine'))

            # Missing or non-positive expected_hours is refused on plan unless --hours is passed
            for bad in ({'objective': 'Click Save'}, {'objective': 'Click Save', 'expected_hours': 0},
                        {'objective': 'Click Save', 'expected_hours': -1}):
                source = Path(directory) / 'bad_hours.json'
                source.write_text(json.dumps({'task': bad}), encoding='utf-8')
                out = io.StringIO()
                with patch.object(sys, 'argv', ['route_judge.py', 'plan', '--dry-run', '--input', str(source)]), \
                        contextlib.redirect_stdout(out):
                    bad_code = router.main()
                self.assertEqual(bad_code, 1)
                self.assertIn('expected_hours', json.loads(out.getvalue()).get('detail', ''))
            code_flag, res_flag, _ = self.run_cli(directory, 'plan', {'objective': 'Click Save'}, '--hours', '0.5')
            self.assertEqual((code_flag, res_flag['decision']['task']['hours']), (0, 0.5))

    def test_hours_come_from_the_same_classification_and_explicit_hours_win(self):
        base = {k: v for k, v in dict(GUI_PROFILE, work_kind='code_implementation',
                                      needs_visual=False, needs_gui=False, local_only=False).items()
                if k != 'axis_demand'}
        estimate = dict(base, expected_hours=3.0)
        task = {'objective': 'Implement the parser module with tests', 'hours_source': 'classifier_estimate', 'expected_hours': None}
        with tempfile.TemporaryDirectory() as directory:
            code, result, calls = self.run_cli(directory, 'plan', task, stdout=self.envelope(estimate))
            self.assertEqual((code, len(calls)), (0, 1))  # one scorer call carries class and hours
            self.assertEqual(result['hours_estimate'], {'hours': 3.0, 'source': 'classifier_estimate'})
            self.assertEqual(result['decision']['task']['hours'], 3.0)
            code, unestimated, _ = self.run_cli(directory, 'plan', task, stdout=self.envelope(base))
            # No default stands in for an hours estimate the classifier did not give.
            self.assertEqual(code, 1)
            self.assertIn('expected_hours is required', unestimated['detail'])
            _, explicit, _ = self.run_cli(directory, 'plan', {'objective': task['objective'], 'expected_hours': 0.25},
                                          stdout=self.envelope(estimate))
            self.assertNotIn('hours_estimate', explicit)
            self.assertEqual(explicit['decision']['task']['hours'], 0.25)
        for bad in ({'expected_hours': 'x'}, {'expected_hours': True}):
            with self.assertRaises(ValueError):
                task_intake.validate_profile(dict(base, **bad))
        self.assertEqual(task_intake.validate_profile(dict(base, expected_hours=999))['expected_hours'], 24.0)
        self.assertNotIn('expected_hours', task_intake.validate_profile(base))

    def test_profile_local_only_is_trusted_and_caller_values_win(self):
        profile = dict(GUI_PROFILE, work_kind='code_implementation', needs_visual=False,
                       needs_gui=False, primary_level=2,
                       axis_demand={'creativity': 1, 'coding': 2, 'aesthetics': 0, 'tool_operation': 1})
        with tempfile.TemporaryDirectory() as directory:
            unavail_code, unavail_res, _ = self.run_cli(directory, 'plan', {'objective': 'Explore local logs'}, stdout='not json')
            self.assertEqual((unavail_code, unavail_res['choice'], unavail_res['error'], unavail_res['caller_action']),
                             (2, 'none', 'scorer_unavailable', 'parent_autonomous_choice'))
        # Short/edit-only reasons intentionally fail the caller validator but remain trusted classifier output.
        cases = [
            ({}, 'Uses the local pytest environment.', True, 'Uses the local pytest environment.'),
            ({'local_only': None}, 'local', True, 'local'),
            ({}, 'in-place edit', True, 'in-place edit'),
            ({}, '', True, 'Task profile requires execution on this machine.'),
            ({'local_only': False}, 'Uses the local pytest environment.', False, None),
            ({'local_only': True, 'local_only_reason': 'Needs local ssh-agent credentials'},
             'Classifier reason', True, 'Needs local ssh-agent credentials'),
        ]
        for needs, reason, expected_local, expected_reason in cases:
            with self.subTest(needs=needs, reason=reason), tempfile.TemporaryDirectory() as directory:
                task = {'objective': 'Write CSV parser', 'needs': needs}
                code, result, _ = self.run_cli(directory, 'plan', task,
                                              stdout=self.envelope(dict(profile, reason=reason)))
                self.assertEqual(code, 0)
                self.assertEqual(result['scorer']['source'], 'system_one')
                self.assertEqual(result['decision']['task']['difficulty_source'], 'profile')
                self.assertNotEqual(result['choice'], 'none')
                self.assertEqual(dot_admitted(result), not expected_local)
                if expected_local:
                    self.assertNotEqual(result['choice'], 'dot_text')
                applied = [a for a in result['alerts'] if a['code'] == 'profile_local_only_applied']
                self.assertEqual(bool(applied), needs.get('local_only') is None)
                if applied:
                    self.assertEqual((applied[0]['severity'], applied[0]['reason']), ('info', expected_reason))
                self.assertNotIn('local_only_ignored_without_reason', [a['code'] for a in result['alerts']])
                ledger = json.loads((Path(directory) / 'ledger.jsonl').read_text(encoding='utf-8').splitlines()[-1])
                self.assertEqual(ledger.get('local_only_reason'), expected_reason)
                self.assertEqual(bool(ledger['local_only']), expected_local)
                with router.dispatch_registry.transaction() as connection:
                    decision = router.dispatch_registry.get(connection, 'decisions', result['decision_id'])
                self.assertEqual(decision['allocation_task']['needs']['local_only'], expected_local)
                self.assertEqual(decision['allocation_task']['needs'].get('local_only_reason'), expected_reason)
                self.assertEqual(decision['task']['needs'], needs)


JEV_RESPONSE = {
    'model': 'jev-2026-10-01',
    'answers': {
        'work_kind': {'type': 'choice', 'choice': 'gui_control', 'confidence': 0.8,
                      'probabilities': {'gui_control': 0.85, 'live_loop': 0.15}},
        'primary_level': {'type': 'score', 'score': 1.4, 'confidence': 0.5,
                          'probabilities': {'0': 0.2, '1': 0.4, '2': 0.4}},
        'secondary_kind': {'type': 'choice', 'choice': 'none', 'confidence': 0.9,
                           'probabilities': {'none': 0.9, 'live_loop': 0.1}},
        'secondary_level': {'type': 'score', 'score': 0.1, 'confidence': 0.9,
                            'probabilities': {'0': 0.9, '1': 0.1, '2': 0.0, '3': 0.0}},
        'access': {'type': 'choice', 'choice': 'write', 'confidence': 0.8,
                   'probabilities': {'write': 0.8, 'read-only': 0.2}},
        'local_only': {'type': 'noul', 'noul': 0.7},
        'needs_visual': {'type': 'noul', 'noul': 0.1},
        'wants_fast': {'type': 'noul', 'noul': 0.05},
        'task_count': {'type': 'choice', 'choice': '1'},
    },
    'usage': {'input_tokens': 900, 'output_tokens': 40},
}
JEV_PROFILE = {
    'work_kind': 'gui_control',
    'primary_level': 2,
    'secondary_kind': 'none',
    'secondary_level': 0,
    'local_only': True,
    'needs_visual': True,
    'wants_fast': False,
    'needs_gui': True,
    'read_only_review': False,
    'access': 'write',
    'task_count': '1',
    'multi_task': False,
    'axis_demand': {'creativity': 0, 'coding': 1, 'aesthetics': 1, 'tool_operation': 3},
}
SECRET = 'test-key-0000000000000000'
# The same answers in the Gemini Decisions API's documented shape (https://aistudio.google.com/docs/decisions-api):
# a type and confidence per answer, a legend on score answers, input-token usage, and no model name.
DECISIONS_RESPONSE = {
    'answers': {name: {'confidence': 0.5, **answer, **({'legend': {'0': 'Level 0', '1': 'Level 1'}} if answer['type'] == 'score' else {})}
                for name, answer in JEV_RESPONSE['answers'].items()},
    'usage': {'total_input_tokens': 900, 'input_tokens_by_modality': [{'modality': 'text', 'tokens': 900}],
              'total_output_tokens': 0, 'total_tokens': 900},
}


class SelectableScorerTests(unittest.TestCase):
    """Contract tests for the user-selectable System-One scorers (config, Jev, Claude, Codex, custom API, heuristic)."""

    def setUp(self):
        self.capabilities = reference_capabilities()
        self.path = temporary_root(self.addCleanup) / 'scorer-config.json'
        env = patch.dict(router.os.environ, {'ROUTER_SCORER_CONFIG': str(self.path), 'ROUTER_SCORER_TYPE': '',
                                             'ROUTER_DISABLE_LIVE_SCORER': '', 'ROUTER_SCORE_CACHE': 'off'})
        env.start()
        self.addCleanup(env.stop)
        no_user_env = patch.object(env_vars, 'user_env', return_value=None)
        no_user_env.start()
        self.addCleanup(no_user_env.stop)
        self.task = {'objective': 'Click Save in the local app and take a screenshot', 'work_kind': 'live_loop',
                     'chain': {'id': 'x'}}

    def score(self, config, post=None, run=None, prefix=('agy.exe',), cli=('cli.exe',)):
        calls = {'post': [], 'run': []}

        def fake_post(url, body, api_key, timeout, key_header=None):
            calls['post'].append({'url': url, 'body': body, 'key': api_key, 'timeout': timeout, 'key_header': key_header})
            if isinstance(post, Exception):
                raise post
            return post

        def fake_run(command, cwd, prompt=None, timeout=120, account=None):
            calls['run'].append({'command': command, 'cwd': cwd, 'stdin': prompt})
            return run(command, cwd) if callable(run) else (0, json.dumps({'status': 'SUCCESS', 'structured_output': GUI_PROFILE}), '')

        with patch.object(scorer_client, 'post_json', side_effect=fake_post), \
                patch.object(subscription_cli, 'run_process', side_effect=fake_run), \
                patch.object(scorer_client, 'scorer_prefix', return_value=list(prefix)), \
                patch.object(subscription_cli, 'cli_prefix', return_value=list(cli)):
            profile, meta = scorer_client.score_task(self.task, self.capabilities, scorer_config=config)
        return profile, meta, calls

    def test_config_load_save_env_override_and_masking(self):
        self.assertEqual(scorer_client.load_scorer_config(), {'scorer_type': 'claude_haiku_low', 'fallback_to_agy': True})
        doc = scorer_client.save_scorer_config('jev', api_key=f'  {SECRET} ')
        self.assertEqual(doc, {'scorer_type': 'jev', 'jev_api_key': SECRET})
        self.assertEqual(json.loads(self.path.read_text(encoding='utf-8'))['jev_api_key'], SECRET)
        # None keeps the stored key, '' clears it; base_url/model are stored per scorer type
        scorer_client.save_scorer_config('jev', api_key=None, base_url='https://api.typesafe.ai/', model='jev-latest')
        config = scorer_client.load_scorer_config()
        self.assertEqual((config['scorer_type'], config['jev_api_key'], config['jev_base_url']), ('jev', SECRET, 'https://api.typesafe.ai'))
        scorer_client.save_scorer_config('openai_compatible', api_key='test-custom-key', base_url='http://127.0.0.1:8080/v1',
                                  model='qwen3-4b', fallback_to_agy=False)
        config = scorer_client.load_scorer_config()
        self.assertEqual((config['jev_api_key'], config['api_key'], config['fallback_to_agy']), (SECRET, 'test-custom-key', False))
        # A Jev key is never resolved from a custom endpoint's api_key, and the custom key never from Jev's
        self.assertEqual(scorer_client.scorer_api_settings('jev', {'api_key': 'sk-custom', 'base_url': 'https://x.example'})[0], None)
        self.assertEqual(scorer_client.scorer_api_settings('jev', {'api_key': 'sk-bare'})[:2], ('sk-bare', 'file'))
        with patch.object(env_vars, 'user_env', side_effect=lambda name: 'env-key' if name == 'JEV_API_KEY' else None):
            self.assertEqual(scorer_client.scorer_api_settings('jev', {})[:4], ('env-key', 'env:JEV_API_KEY', 'https://api.typesafe.ai', 'jev-latest'))
        scorer_client.save_scorer_config('jev', api_key='')
        self.assertNotIn('jev_api_key', scorer_client.load_scorer_config())
        for bad in (lambda: scorer_client.save_scorer_config('gpt-9'),
                    lambda: scorer_client.save_scorer_config('openai_compatible', base_url='', model=''),
                    lambda: scorer_client.save_scorer_config('jev', base_url='http://api.typesafe.ai'),
                    lambda: scorer_client.save_scorer_config('heuristic')):
            with self.assertRaises(ValueError):
                bad()
        with patch.dict(router.os.environ, {'ROUTER_SCORER_TYPE': 'heuristic'}):
            config = scorer_client.load_scorer_config()
            self.assertEqual((config['scorer_type'], 'heuristic' in config['config_error']), ('claude_haiku_low', True))
        with patch.dict(router.os.environ, {'ROUTER_SCORER_TYPE': 'nope'}):
            config = scorer_client.load_scorer_config()
            self.assertEqual((config['scorer_type'], 'nope' in config['config_error']), ('claude_haiku_low', True))
        self.path.write_text('not json', encoding='utf-8')
        self.assertEqual(scorer_client.load_scorer_config()['scorer_type'], 'claude_haiku_low')
        self.assertEqual((scorer_client.mask_secret(SECRET), scorer_client.mask_secret('short'), scorer_client.mask_secret(None)),
                         ('test…0000', '••••', ''))
        view = scorer_client.scorer_status({'scorer_type': 'jev', 'jev_api_key': SECRET})
        self.assertEqual((view['jev']['api_key'], view['jev']['key_source']), ('test…0000', 'file'))
        self.assertNotIn(SECRET, json.dumps(view))

    def test_jev_reads_the_whole_brief_and_answers_multi_task_and_task_count(self):
        objective = 'Two things: (1) fix the parser crash; (2) translate the README into Japanese.'
        request = scorer_client.build_jev_request({'objective': objective}, self.capabilities)
        self.assertEqual(request['state']['task'], {'objective': objective})
        questions = request['questions']
        self.assertNotIn('multi_task', questions)
        self.assertEqual(questions['task_count']['type'], 'choice')
        self.assertEqual(list(questions['task_count']['criteria']), ['1', '2', '3', '>3'])
        answers = {**JEV_RESPONSE['answers'], 'task_count': {'choice': '2'}}
        profile = scorer_client.parse_profile_answers({'answers': answers})
        self.assertEqual((profile['multi_task'], profile['task_count']), (True, '2'))

    def test_jev_request_matches_typesafe_systemone_contract(self):
        request = scorer_client.build_jev_request(self.task, self.capabilities)
        self.assertEqual((request['model'], set(request['state'])), ('jev-latest', {'task', 'rubric_notes'}))
        self.assertEqual(request['state']['task'], {'objective': self.task['objective']})
        self.assertIn('access_and_local_only', request['state']['rubric_notes'])
        questions = request['questions']
        self.assertEqual(set(questions), {'work_kind', 'primary_level', 'secondary_kind', 'secondary_level',
                                          'access', 'local_only', 'needs_visual', 'wants_fast',
                                          'effort_band', 'task_count'})
        routes = set(self.capabilities['routes']) - {'other_or_mixed'}
        self.assertEqual((questions['work_kind']['type'], set(questions['work_kind']['criteria'])), ('choice', routes))
        self.assertTrue(all(isinstance(v, str) and v for v in questions['work_kind']['criteria'].values()))
        self.assertEqual((questions['secondary_kind']['type'], set(questions['secondary_kind']['criteria'])),
                         ('choice', {'none'} | routes))
        self.assertEqual((questions['access']['type'], set(questions['access']['criteria'])), ('choice', {'read-only', 'write'}))
        for name, (low, high) in task_intake.PROFILE_INTEGERS.items():
            q = questions[name]
            self.assertEqual((q['type'], len(q['criteria']), isinstance(q['criteria'], list)), ('score', high - low + 1, True), name)
            self.assertLessEqual(len(q['criteria']), 10)
        for name in ('needs_visual', 'local_only', 'wants_fast'):
            self.assertEqual(questions[name]['type'], 'noul')
            self.assertNotIn('criteria', questions[name])

    def test_jev_response_parsing_maps_every_dimension(self):
        profile = scorer_client.parse_profile_answers(JEV_RESPONSE)
        self.assertEqual(profile, JEV_PROFILE)
        self.assertEqual(task_intake.validate_profile(profile), profile)
        # Scores clamp into range; single-domain vs composite secondary_kind reconciles cleanly; gui_control implies needs_gui/visual
        answers = {k: (dict(v) if isinstance(v, dict) else v) for k, v in JEV_RESPONSE['answers'].items()}
        answers['primary_level']['score'] = 9
        answers['secondary_kind'] = {'choice': 'live_loop'}
        answers['secondary_level'] = {'score': 1.8}
        answers['needs_visual'] = {'type': 'noul', 'noul': 0.49}
        answers['local_only'] = 0.5
        answers['work_kind'] = 'code_implementation'
        answers['access'] = {'choice': 'write'}
        p2 = scorer_client.parse_profile_answers({'answers': answers})
        self.assertEqual((p2['primary_level'], p2['secondary_kind'], p2['secondary_level'],
                          p2['needs_gui'], p2['needs_visual'], p2['local_only'], p2['work_kind'], p2['access']),
                         (3, 'live_loop', 2, False, False, True, 'code_implementation', 'write'))
        self.assertEqual(p2['axis_demand'], {'creativity': 1, 'coding': 3, 'aesthetics': 0, 'tool_operation': 2})
        self.assertNotIn('reason', p2)
        # Same secondary_kind as work_kind or secondary_level=0 on composite reconciles deterministically
        same_sec = scorer_client.parse_profile_answers(
            {'answers': {**answers, 'secondary_kind': {'choice': 'code_implementation'}, 'secondary_level': {'score': 2.0}}})
        self.assertEqual((same_sec['secondary_kind'], same_sec['secondary_level']), ('none', 0))
        zero_sec = scorer_client.parse_profile_answers(
            {'answers': {**answers, 'secondary_kind': {'choice': 'live_loop'}, 'secondary_level': {'score': 0.0}}})
        self.assertEqual((zero_sec['secondary_kind'], zero_sec['secondary_level']), ('live_loop', 1))
        for broken in ({'answers': {**answers, 'work_kind': {'choice': 'make_coffee'}}},
                       {'answers': {**answers, 'secondary_kind': {'choice': 'vibes'}}},
                       {'answers': {**answers, 'access': {'choice': 'sudo'}}},
                       {'answers': {**answers, 'work_kind': 'code_implementation', 'local_only': 0.9, 'access': {'choice': 'read-only'}}},
                       {'answers': {**answers, 'work_kind': 'code_audit', 'access': {'choice': 'write'}}},
                       {'answers': {k: v for k, v in answers.items() if k != 'primary_level'}},
                       {'answers': {k: v for k, v in answers.items() if k != 'secondary_kind'}},
                       {'answers': {k: v for k, v in answers.items() if k != 'wants_fast'}},
                       {'answers': {k: v for k, v in answers.items() if k != 'access'}},
                       {'answers': {**answers, 'primary_level': {'score': float('nan')}}},
                       {'error': 'x'}):
            with self.assertRaises(ValueError):
                scorer_client.parse_profile_answers(broken)
        for bad_prof in (
            {k: v for k, v in GUI_PROFILE.items() if k != 'wants_fast'},
            {k: v for k, v in GUI_PROFILE.items() if k != 'access'},
            {**GUI_PROFILE, 'work_kind': 'code_audit', 'read_only_review': True, 'access': 'write'},
        ):
            with self.assertRaises(ValueError):
                task_intake.validate_profile(bad_prof)

    def test_jev_answers_effort_band_in_the_same_request(self):
        answers = {**JEV_RESPONSE['answers'], 'effort_band': {'choice': 'multi_file'}}
        profile = scorer_client.parse_profile_answers({'answers': answers})
        self.assertEqual(profile['expected_hours'], 2.0)
        for value in ('minutes', None):  # unknown or missing answers omit expected_hours instead of inventing one
            plain = scorer_client.parse_profile_answers({'answers': {**answers, 'effort_band': {'choice': value}}})
            self.assertNotIn('expected_hours', plain)

    def test_jev_scorer_dispatch_and_fallbacks(self):
        profile, meta, calls = self.score({'scorer_type': 'jev', 'jev_api_key': SECRET}, post=JEV_RESPONSE)
        self.assertEqual(profile, JEV_PROFILE)
        self.assertEqual((meta['source'], meta['scorer_type'], meta['provider'], meta['model']),
                         ('system_one', 'jev', 'typesafe', 'jev-2026-10-01'))
        self.assertEqual((calls['post'][0]['url'], calls['post'][0]['key'], calls['post'][0]['key_header'], calls['run']),
                         ('https://api.typesafe.ai/v1/systemone', SECRET, None, []))
        self.assertNotIn('chain', json.dumps(calls['post'][0]['body']))
        # Missing key: jev is absent on this host, so the next scorer in order answers and nothing counts as a failure
        _, meta, calls = self.score({'scorer_type': 'jev', 'fallback_to_agy': True})
        self.assertEqual((meta['source'], meta['scorer_type'], calls['post']), ('system_one', 'claude_haiku_low', []))
        self.assertNotIn('attempts', meta)
        # fallback_to_agy off, or all scorers failing: returns scorer_unavailable (None profile) so parent session chooses
        profile, meta, calls = self.score({'scorer_type': 'jev', 'fallback_to_agy': False})
        self.assertEqual((profile, meta['source'], meta['configured_scorer'], calls['run']), (None, 'scorer_unavailable', 'jev', []))
        self.assertNotIn('attempts', meta)
        profile, meta, _ = self.score({'scorer_type': 'jev', 'jev_api_key': SECRET},
                                      post=scorer_client.ScorerError('Scorer API answered HTTP 401.'), run=lambda c, d: (1, '', ''))
        self.assertEqual((profile, meta['source'], [a['scorer_type'] for a in meta['attempts']]),
                         (None, 'scorer_unavailable', ['jev', 'claude_haiku_low', 'codex_terra_low', 'agy_flash_low']))
        self.assertEqual(meta['fallback_detail'], 'Scorer API answered HTTP 401.')
        self.assertNotIn(SECRET, json.dumps(meta))
        with patch.dict(router.os.environ, {'ROUTER_DISABLE_LIVE_SCORER': '1'}):
            prof_dis, meta, calls = self.score({'scorer_type': 'jev', 'jev_api_key': SECRET, 'fallback_to_agy': False}, post=JEV_RESPONSE)
        self.assertEqual((prof_dis, meta['source'], calls['post']), (None, 'scorer_unavailable', []))

    def test_post_json_reports_http_failures_without_bodies(self):
        error = router.urllib.error.HTTPError('https://api.typesafe.ai/v1/systemone', 401, 'Unauthorized', {},
                                             io.BytesIO(b'{"detail":"bad key ts-live-0123"}'))
        with patch.object(router.urllib.request, 'urlopen', side_effect=error):
            with self.assertRaises(scorer_client.ScorerError) as caught:
                scorer_client.post_json('https://api.typesafe.ai/v1/systemone', {}, SECRET, 5)
        self.assertEqual(str(caught.exception), 'Scorer API answered HTTP 401.')
        with patch.object(router.urllib.request, 'urlopen', side_effect=router.urllib.error.URLError('dns')):
            with self.assertRaisesRegex(scorer_client.ScorerError, 'unreachable'):
                scorer_client.post_json('https://api.typesafe.ai/v1/systemone', {}, SECRET, 5)

        class Response(io.BytesIO):
            def __enter__(self):
                return self

            def __exit__(self, *exc):
                return False
        seen = {}

        def urlopen(request, timeout):
            seen.update(url=request.full_url, auth=request.get_header('Authorization'), body=json.loads(request.data), timeout=timeout)
            return Response(json.dumps(JEV_RESPONSE).encode('utf-8'))
        with patch.object(router.urllib.request, 'urlopen', side_effect=urlopen):
            self.assertEqual(scorer_client.post_json('https://api.typesafe.ai/v1/systemone', {'model': 'jev-latest'}, SECRET, 7), JEV_RESPONSE)
        self.assertEqual(seen, {'url': 'https://api.typesafe.ai/v1/systemone', 'auth': f'Bearer {SECRET}',
                                'body': {'model': 'jev-latest'}, 'timeout': 7})

    def test_decisions_request_follows_the_documented_contract(self):
        # Request reference and limits: https://aistudio.google.com/docs/decisions-api
        task = {**self.task, 'context': '重置前冲刺'}
        request = scorer_client.build_decisions_request(task, self.capabilities)
        self.assertEqual((set(request), request['model']), ({'model', 'input', 'questions'}, 'gemini-decision-1'))
        # A JSON object goes in as text, whole, and it is the state the other question API gets.
        jev = scorer_client.build_jev_request(task, self.capabilities)
        self.assertIsInstance(request['input'], str)
        self.assertIn('重置前冲刺', request['input'])
        self.assertEqual((json.loads(request['input']), request['questions']), (jev['state'], jev['questions']))
        self.assertTrue(1 <= len(request['questions']) <= 1000)
        for key, question in request['questions'].items():
            self.assertRegex(key, r'\A[A-Za-z0-9_-]{1,64}\Z')
            # The model never sees the key, so the instructions carry the whole question.
            self.assertTrue(isinstance(question['instructions'], str) and question['instructions'].strip(), key)
            criteria = question.get('criteria')
            if question['type'] == 'choice':
                self.assertTrue(2 <= len(criteria) <= 255, key)
                for option, description in criteria.items():
                    self.assertTrue(1 <= len(option) <= 128 and option == option.strip() and option.isprintable(), (key, option))
                    self.assertTrue(isinstance(description, str) and description, (key, option))
            elif question['type'] == 'score':
                self.assertTrue(isinstance(criteria, list) and 2 <= len(criteria) <= 255, key)
                self.assertTrue(all(isinstance(level, str) and level for level in criteria), key)
            else:
                self.assertEqual((question['type'], criteria), ('noul', None), key)

    def test_gemini_decision_scorer_dispatch_key_isolation_and_order(self):
        config = {'scorer_type': 'gemini_decision', 'gemini_api_key': SECRET}
        profile, meta, calls = self.score(config, post=DECISIONS_RESPONSE)
        self.assertEqual(profile, JEV_PROFILE)
        # The answer names no model, so the configured one is reported.
        self.assertEqual((meta['source'], meta['scorer_type'], meta['provider'], meta['model']),
                         ('system_one', 'gemini_decision', 'google', 'gemini-decision-1'))
        call = calls['post'][0]
        self.assertEqual((call['url'], call['key'], call['key_header'], calls['run']),
                         ('https://generativelanguage.googleapis.com/v1beta/decisions', SECRET, 'x-goog-api-key', []))
        self.assertNotIn('chain', json.dumps(call['body']))
        _, meta, calls = self.score({**config, 'gemini_base_url': 'https://proxy.example.com', 'gemini_model': 'gemini-decision-2'},
                                    post=DECISIONS_RESPONSE)
        self.assertEqual((calls['post'][0]['url'], calls['post'][0]['body']['model'], meta['model']),
                         ('https://proxy.example.com/v1beta/decisions', 'gemini-decision-2', 'gemini-decision-2'))
        # One API's key is never resolved for another API.
        self.assertEqual(scorer_client.scorer_api_settings('gemini_decision', {'api_key': 'sk-custom', 'jev_api_key': 'ts-key'})[:2],
                         (None, None))
        for other in ('jev', 'openai_compatible'):
            self.assertIsNone(scorer_client.scorer_api_settings(other, {'gemini_api_key': SECRET})[0])
        # A key alone, in the file or the environment, never enlists the billed API: only the chosen type runs it.
        with patch.object(env_vars, 'user_env', side_effect=lambda name: 'env-key' if name == 'GEMINI_API_KEY' else None):
            self.assertEqual(scorer_client.scorer_api_settings('gemini_decision', {}),
                             ('env-key', 'env:GEMINI_API_KEY', 'https://generativelanguage.googleapis.com', 'gemini-decision-1'))
            self.assertNotIn('gemini_decision',
                             scorer_client.scorer_attempt_order({'scorer_type': 'claude_haiku_low', 'fallback_to_agy': True})[1])
        self.assertNotIn('gemini_decision', scorer_client.scorer_attempt_order(
            {'scorer_type': 'claude_haiku_low', 'gemini_api_key': SECRET, 'fallback_to_agy': True})[1])
        configured, order = scorer_client.scorer_attempt_order({**config, 'fallback_to_agy': True})
        self.assertEqual((configured, order[0], order.count('gemini_decision')), ('gemini_decision', 'gemini_decision', 1))
        self.assertEqual(scorer_client.scorer_attempt_order({**config, 'fallback_to_agy': False}),
                         ('gemini_decision', ['gemini_decision']))
        # Missing key: absent on this host, so the next scorer in order answers and nothing counts as a failure.
        _, meta, calls = self.score({'scorer_type': 'gemini_decision', 'fallback_to_agy': True})
        self.assertEqual((meta['source'], meta['scorer_type'], calls['post']), ('system_one', 'claude_haiku_low', []))
        self.assertNotIn('attempts', meta)
        self.assertIn('GEMINI_API_KEY', meta['skipped'][0]['detail'])
        # A refused call (HTTP 400: a limit, a blocked input, a bad key) is a failure, never a profile; no key in the metadata.
        profile, meta, _ = self.score({**config, 'fallback_to_agy': False}, post=scorer_client.ScorerError('Scorer API answered HTTP 400.'))
        self.assertEqual((profile, meta['source'], meta['fallback_detail']), (None, 'scorer_unavailable', 'Scorer API answered HTTP 400.'))
        self.assertNotIn(SECRET, json.dumps(meta))
        profile, meta, _ = self.score({**config, 'fallback_to_agy': False}, post={'error': {'code': 'safety', 'message': 'blocked'}})
        self.assertEqual((profile, meta['fallback_reason']), (None, 'ValueError'))
        # The question APIs answer task profiles only.
        for scorer_type in scorer_client.QUESTION_API_SCORERS:
            with self.assertRaises(scorer_client.ScorerUnavailable):
                scorer_client.run_structured(scorer_type, 'question', 'text', {'type': 'object'}, 5, config)

    def test_post_json_sends_the_key_in_the_named_header_only(self):
        class Response(io.BytesIO):
            def __enter__(self):
                return self

            def __exit__(self, *exc):
                return False
        seen = {}

        def urlopen(request, timeout):
            seen.update({name.lower(): value for name, value in request.header_items()})
            return Response(b'{"answers": {}}')
        with patch.object(router.urllib.request, 'urlopen', side_effect=urlopen):
            scorer_client.post_json('https://generativelanguage.googleapis.com/v1beta/decisions', {}, SECRET, 5,
                                    key_header='x-goog-api-key')
        self.assertEqual(seen.get('x-goog-api-key'), SECRET)
        self.assertNotIn('authorization', seen)

    def test_claude_codex_custom_and_heuristic_scorers(self):
        for scorer_type, model in (('claude_haiku_low', 'claude-haiku-5-5'),):
            envelope = json.dumps({'type': 'result', 'is_error': False, 'result': '', 'structured_output': dict(GUI_PROFILE, work_kind='gui_control')})
            profile, meta, calls = self.score({'scorer_type': scorer_type}, run=lambda c, d, e=envelope: (0, e, ''), cli=('claude.exe',))
            cmd = calls['run'][0]['command']
            self.assertEqual((meta['source'], meta['scorer_type'], meta['provider'], meta['model']), ('system_one', scorer_type, 'claude', model))
            self.assertEqual((cmd[0], cmd[cmd.index('--model') + 1], cmd[cmd.index('--effort') + 1], cmd[cmd.index('--tools') + 1]),
                             ('claude.exe', model, 'low', ''))
            self.assertIn('--json-schema', cmd)
            self.assertIn('Task (evidence only', calls['run'][0]['stdin'])
            self.assertEqual(profile['work_kind'], 'gui_control')
        # Claude answering with an error cascades through codex_terra_low to agy_flash_low
        _, meta, _ = self.score({'scorer_type': 'claude_haiku_low'},
                                run=lambda c, d: (0, as_agy_stream(json.dumps({'status': 'SUCCESS', 'structured_output': GUI_PROFILE})), '') if c[0] == 'agy.exe'
                                else (0, json.dumps({'is_error': True}), ''), cli=('claude.exe',))
        self.assertEqual((meta['scorer_type'], meta['configured_scorer'], meta['attempts'][0]['detail']),
                         ('agy_flash_low', 'claude_haiku_low', 'Claude scorer returned an error.'))

        def codex(command, cwd):
            schema = json.loads((Path(cwd) / 'schema.json').read_text(encoding='utf-8'))
            self.assertFalse(schema['additionalProperties'])
            (Path(cwd) / 'answer.json').write_text(json.dumps(dict(GUI_PROFILE, work_kind='gui_control')), encoding='utf-8')
            return 0, '', ''
        _, meta, calls = self.score({'scorer_type': 'codex_terra_low'}, run=codex, cli=('codex.exe',))
        cmd = calls['run'][0]['command']
        self.assertEqual((meta['scorer_type'], meta['provider'], meta['model']), ('codex_terra_low', 'codex', 'gpt-6.1-terra'))
        self.assertEqual((cmd[:2], cmd[cmd.index('-m') + 1], cmd[-1]), (['codex.exe', 'exec'], 'gpt-6.1-terra', '-'))
        self.assertIn('model_reasoning_effort="low"', cmd)

        self.assertEqual([scorer_client.openai_chat_url(u) for u in ('https://x.ai', 'https://x.ai/v1/', 'https://x.ai/api/chat/completions')],
                         ['https://x.ai/v1/chat/completions', 'https://x.ai/v1/chat/completions', 'https://x.ai/api/chat/completions'])
        content = 'Here you go:\n' + json.dumps(dict(GUI_PROFILE, work_kind='gui_control'))
        reply = {'choices': [{'message': {'role': 'assistant', 'content': content}}]}
        custom = {'scorer_type': 'openai_compatible', 'api_key': 'sk-custom', 'base_url': 'https://llm.example.com/v1', 'model': 'qwen3-4b'}
        profile, meta, calls = self.score(custom, post=reply)
        body = calls['post'][0]['body']
        self.assertEqual((meta['scorer_type'], meta['provider'], meta['model'], profile['work_kind']),
                         ('openai_compatible', 'openai_compatible', 'qwen3-4b', 'gui_control'))
        self.assertEqual((calls['post'][0]['url'], calls['post'][0]['key'], body['model'], body['response_format']),
                         ('https://llm.example.com/v1/chat/completions', 'sk-custom', 'qwen3-4b', {'type': 'json_object'}))
        self.assertTrue(body['messages'][0]['content'].startswith(scorer_client.SCORER_PROMPT))
        prof_no_key, meta, calls = self.score({**custom, 'api_key': '', 'fallback_to_agy': False}, post=reply)
        self.assertEqual((prof_no_key, meta['source'], calls['post']), (None, 'scorer_unavailable', []))
        prof_empty, meta, _ = self.score({**custom, 'fallback_to_agy': False}, post={'choices': []})
        self.assertEqual((prof_empty, meta['source'], meta['fallback_reason']), (None, 'scorer_unavailable', 'ValueError'))

        # Priority cascade order: jev (when key present) -> claude_haiku_low -> codex_terra_low -> agy_flash_low
        self.assertEqual(scorer_client.scorer_attempt_order({'scorer_type': 'claude_haiku_low', 'fallback_to_agy': True}),
                         ('claude_haiku_low', ['claude_haiku_low', 'codex_terra_low', 'agy_flash_low']))
        self.assertEqual(scorer_client.scorer_attempt_order({'scorer_type': 'claude_haiku_low', 'jev_api_key': SECRET, 'fallback_to_agy': True}),
                         ('claude_haiku_low', ['jev', 'claude_haiku_low', 'codex_terra_low', 'agy_flash_low']))

    def test_score_task_reads_the_configured_scorer_and_scorer_cli(self):
        scorer_client.save_scorer_config('claude_haiku_low')

        def cli(*argv, stdin=None):
            out = io.StringIO()
            with patch.object(sys, 'argv', ['route_judge.py', 'scorer', *argv]), contextlib.redirect_stdout(out), \
                    patch.object(sys, 'stdin', io.StringIO(stdin or '')):
                code = router.main()
            return code, json.loads(out.getvalue()), out.getvalue()
        code, shown, _ = cli()
        self.assertEqual((code, shown['action'], shown['scorer_type'], shown['config_path']), (0, 'show', 'claude_haiku_low', str(self.path)))
        code, saved, text = cli('--action', 'set', '--type', 'jev', '--api-key', '-', stdin=SECRET + '\n')
        self.assertEqual((code, saved['scorer_type'], saved['jev']['api_key'], saved['jev']['key_source']), (0, 'jev', 'test…0000', 'file'))
        self.assertNotIn(SECRET, text)
        self.assertEqual(json.loads(self.path.read_text(encoding='utf-8'))['jev_api_key'], SECRET)
        code, failed, _ = cli('--action', 'set')
        self.assertEqual((code, failed['detail']), (1, 'scorer set requires --type.'))
        with patch.object(scorer_client, 'post_json', side_effect=scorer_client.ScorerError('Scorer API answered HTTP 401.')), \
                patch.object(subscription_cli, 'cli_prefix', side_effect=RuntimeError('cli missing')), \
                patch.object(scorer_client, 'scorer_prefix', side_effect=RuntimeError('agy missing')):
            code, tested, _ = cli('--action', 'test')
        self.assertEqual((code, tested['ok'], tested['scorer']['source'], tested['scorer']['fallback_detail']),
                         (1, False, 'scorer_unavailable', 'Scorer API answered HTTP 401.'))
        with patch.object(scorer_client, 'post_json', return_value=JEV_RESPONSE):
            code, tested, _ = cli('--action', 'test')
        self.assertEqual((code, tested['ok'], tested['task_profile']), (0, True, JEV_PROFILE))


class ScorerAccountTests(unittest.TestCase):
    """The cascade keeps its order but runs each CLI scorer on an account this host has; absent ones are skipped."""

    INVENTORY = {'accounts': {
        'claude': {'family': 'claude', 'status': 'disabled', 'home': 'C:/u/.claude'},
        'claude-2': {'family': 'claude', 'status': 'ready', 'home': 'C:/u/.claude-2', 'billing': 'subscription'},
        'llmproxy': {'family': 'codex', 'status': 'ready', 'billing': 'metered'},
        'antigravity': {'family': 'antigravity', 'status': 'ready'},
    }}

    def setUp(self):
        self.capabilities = reference_capabilities()
        env = patch.dict(router.os.environ, {'ROUTER_SCORER_CONFIG': str(temporary_root(self.addCleanup) / 'scorer.json'),
                                             'ROUTER_SCORER_TYPE': '', 'ROUTER_DISABLE_LIVE_SCORER': '',
                                             'ROUTER_SCORE_CACHE': 'off'})
        env.start()
        self.addCleanup(env.stop)
        self.task = {'objective': 'Click Save in the local app and take a screenshot', 'work_kind': 'live_loop'}
        real = patch.object(scorer_client, 'scorer_accounts', side_effect=_real_scorer_accounts)
        real.start()
        self.addCleanup(real.stop)

    def test_lists_every_ready_allowed_subscription_account_of_the_family(self):
        rows = self.INVENTORY
        # LITTLESPRING: ~/.claude signed out, ~/.claude-2 signed in.
        self.assertEqual(scorer_client.scorer_accounts('claude_haiku_low', rows, None), ['claude-2'])
        both = {'accounts': {**rows['accounts'], 'claude': {**rows['accounts']['claude'], 'status': 'ready'}}}
        self.assertEqual(scorer_client.scorer_accounts('claude_haiku_low', both, None), ['claude', 'claude-2'])
        self.assertEqual(scorer_client.scorer_accounts('agy_flash_low', rows, None), ['antigravity'])
        self.assertEqual(scorer_client.scorer_accounts('codex_terra_low', rows, None), [])     # a metered proxy is not one
        self.assertEqual(scorer_client.scorer_accounts('claude_haiku_low', both, {'allowed_accounts': ['claude-2']}), ['claude-2'])
        self.assertEqual(scorer_client.scorer_accounts('jev', rows, None), [None])

    def test_absent_scorers_are_skipped_in_order_without_errors(self):
        used = []

        def fake_run(command, cwd, prompt=None, timeout=120, account=None):
            used.append((command[0], account))
            if command[0] == 'agy.exe':
                return 0, as_agy_stream(json.dumps({'status': 'SUCCESS', 'structured_output': GUI_PROFILE})), ''
            return 0, json.dumps({'structured_output': GUI_PROFILE}), ''

        with patch.object(router.provider_inventory, 'current', return_value=self.INVENTORY), \
                patch.object(router.host_policy, 'load_policy', return_value=None), \
                patch.object(subscription_cli, 'run_process', side_effect=fake_run), \
                patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), \
                patch.object(subscription_cli, 'cli_prefix', return_value=['cli.exe']):
            _, meta = scorer_client.score_task(self.task, self.capabilities, scorer_config={})
        self.assertEqual(used, [('cli.exe', 'claude-2')])
        self.assertEqual((meta['source'], meta['scorer_type'], meta['account']), ('system_one', 'claude_haiku_low', 'claude-2'))
        self.assertNotIn('attempts', meta)
        # Both Claude accounts signed in and the first fails: the next Claude account answers before Codex is tried.
        both = {'accounts': {**self.INVENTORY['accounts'],
                             'claude': {**self.INVENTORY['accounts']['claude'], 'status': 'ready'}}}
        used.clear()
        homes = iter([(1, '', ''), (0, json.dumps({'structured_output': GUI_PROFILE}), '')])
        with patch.object(router.provider_inventory, 'current', return_value=both),                 patch.object(router.host_policy, 'load_policy', return_value=None),                 patch.object(subscription_cli, 'run_process', side_effect=lambda c, d, p=None, t=120, account=None:
                             (used.append((c[0], account)), next(homes))[1]),                 patch.object(subscription_cli, 'cli_prefix', return_value=['cli.exe']):
            _, meta = scorer_client.score_task(self.task, self.capabilities, scorer_config={})
        self.assertEqual(used, [('cli.exe', 'claude'), ('cli.exe', 'claude-2')])
        self.assertEqual((meta['account'], [(a['scorer_type'], a['account']) for a in meta['attempts']]),
                         ('claude-2', [('claude_haiku_low', 'claude')]))
        no_claude = {'accounts': {k: v for k, v in self.INVENTORY['accounts'].items() if not k.startswith('claude')}}
        used.clear()
        with patch.object(router.provider_inventory, 'current', return_value=no_claude), \
                patch.object(router.host_policy, 'load_policy', return_value=None), \
                patch.object(subscription_cli, 'run_process', side_effect=fake_run), \
                patch.object(scorer_client, 'scorer_prefix', return_value=['agy.exe']), \
                patch.object(subscription_cli, 'cli_prefix', return_value=['cli.exe']):
            _, meta = scorer_client.score_task(self.task, self.capabilities, scorer_config={})
        self.assertEqual([c for c, _ in used], ['agy.exe'])        # claude and codex absent: agy, no error recorded
        self.assertEqual(meta['scorer_type'], 'agy_flash_low')
        self.assertNotIn('attempts', meta)

    def test_subscription_env_points_codex_and_claude_clis_at_the_chosen_home(self):
        rows = {'accounts': {'codex-2': {'family': 'codex', 'home': 'C:/u/.codex-2'},
                             'claude-2': {'family': 'claude', 'home': 'C:/u/.claude-2'}}}
        with patch.object(router.provider_inventory, 'current', return_value=rows):
            self.assertEqual(subscription_cli.subscription_env('codex-2')['CODEX_HOME'], 'C:/u/.codex-2')
            self.assertEqual(subscription_cli.subscription_env('claude-2')['CLAUDE_CONFIG_DIR'], 'C:/u/.claude-2')

    def test_compact_rubric_column_union_and_prompt_stays_off_the_command_line(self):
        import subprocess

        rubric_het = {
            'calibration_cases': [
                {'id': 'a', 'objective': 'o1', 'expected_profile': {'work_kind': 'code_implementation', 'primary_level': 2}},
                {'id': 'b', 'objective': 'o2', 'expected_profile': {'work_kind': 'gui_control', 'primary_level': 1, 'needs_visual': True}},
            ]
        }
        compact = scorer_client._compact_rubric_for_prompt(rubric_het)
        self.assertEqual(compact['calibration_columns'][2], ['work_kind', 'primary_level', 'needs_visual'])
        self.assertEqual(compact['calibration_cases'][0][2], ['code_implementation', 2, None])
        self.assertEqual(compact['calibration_cases'][1][2], ['gui_control', 1, True])

        capabilities = reference_capabilities()
        schema = scorer_client.scorer_schema(capabilities)
        prompt = scorer_client.scorer_prompt({
            'objective': 'Implement a live order-book dashboard and run unit tests',
            'context': 'Frontend React + WebSocket module',
            'environment': 'Windows 11, Python 3.13',
        })
        # The limit that remains is the scorer model's context window, not a command line. The smallest window of any
        # System-One scorer is taken as 200k tokens (accounts.json cites 500k for the Codex scorer; the others are not
        # cited, so the floor is a stated assumption); the rubric and calibration cases may use a quarter of it, at the
        # conservative 2 characters per token that CJK text needs.
        context_floor_tokens, share, chars_per_token = 200_000, 0.25, 2
        budget_chars = int(context_floor_tokens * share * chars_per_token)
        self.assertLessEqual(len(prompt), budget_chars,
                             f'the scorer prompt is {len(prompt)} chars; the budget is {budget_chars} '
                             f'({share:.0%} of a {context_floor_tokens}-token context at {chars_per_token} chars/token)')
        # Nothing of the prompt rides the command line: it is the same size whether the rubric holds 1 or 1000 cases.
        small = scorer_client.scorer_stream_command(['agy.cmd'], schema)
        with patch.object(scorer_client, 'load_classifier_cases', return_value={'calibration_cases': [
                {'id': f'c{i}', 'task': 'x' * 500, 'expected_profile': {'work_kind': 'code_audit'}} for i in range(1000)]}):
            big_prompt = scorer_client.scorer_prompt({'objective': 'x'})
        self.assertGreater(len(big_prompt), 500_000)
        self.assertEqual(scorer_client.scorer_stream_command(['agy.cmd'], schema), small)
        self.assertLess(len(subprocess.list2cmdline(small)), 30000)
        self.assertFalse(hasattr(router, 'ARGV_LIMIT_CHARS'))


@unittest.skipUnless(router.os.environ.get('ROUTER_LIVE_AGY') == '1', 'set ROUTER_LIVE_AGY=1 to call the real agy CLI')
class LiveSystemOneSmokeTests(unittest.TestCase):
    # Capture only for the explicit opt-in; ordinary tests never restore host state.
    LIVE_ENV = dict(router.os.environ) if router.os.environ.get('ROUTER_LIVE_AGY') == '1' else {}
    LIVE_HOME = Path.home() if LIVE_ENV else None
    TASKS = {
        'gui_click': ({'objective': 'Click the Save button in the Settings dialog of the local desktop app and take a screenshot to confirm.',
                       'work_kind': 'gui_control', 'needs': {'local_only': True, 'local_only_reason': 'local desktop app screen'}},
                      lambda p: p['primary_level'] <= 2 and p['needs_gui']),
        'one_file_edit': ({'objective': 'In tray/deploy.py change the default retry count constant from 3 to 5 and update the one unit test that asserts it.',
                           'work_kind': 'code_implementation', 'needs': {'local_only': True, 'local_only_reason': 'local pytest suite'}},
                          lambda p: p['primary_level'] <= 2 and not p['needs_gui']),
        'core_refactor': ({'objective': 'Refactor the order-matching engine across the core library, the risk module and the gateway so all three '
                                        'share one lock-free event queue, preserving latency and ordering invariants, and migrate every caller.',
                           'needs': {'local_only': True, 'local_only_reason': 'local repository and benchmarks'}},
                          lambda p: p['primary_level'] >= 2 and p['axis_demand']['coding'] >= 2),
    }
    TIMEOUT = 60

    def test_real_scores_and_plans(self):
        capabilities = reference_capabilities()
        for name, (task, plausible) in self.TASKS.items():
            with self.subTest(task=name), patch.dict(router.os.environ, self.LIVE_ENV, clear=True), \
                    patch.object(Path, 'home', return_value=self.LIVE_HOME):
                profile, meta = scorer_client.score_task(task, capabilities, timeout=self.TIMEOUT,
                                                   scorer_config={'scorer_type': 'agy_flash_low'})
            self.assertEqual(meta['source'], 'system_one', (name, meta))
            self.assertTrue(plausible(profile), (name, profile))


if __name__ == '__main__':
    unittest.main()
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall